College Basketball Nation: Selection Committee

NCAA's RPI line as tired as ever

February, 16, 2012
This week, the always-excellent Scott Van Pelt unleashed a rather cathartic rant about the essential uselessness of the Ratings Percentage Index, or RPI. The points Van Pelt made are the ones that must always be made about the RPI:

Van Pelt
Van Pelt
1. The RPI is older than the compact disc, and roughly as technologically advanced.

2. The RPI's methodology (who you beat plus who your opponents beat plus who your opponents' opponents beat) is really dumb. There are countless rankings systems that provide a much more realistic methodology, including the numbers generated and maintained by Ken Pomeroy and Jeff Sagarin, Ken Massey, the big brains that built the LRMC and, most recently, the ESPN Analytics Group's new Basketball Power Index, which is, after a mere week of existence, already five or six times better at evaluating bubble teams than is the RPI.

3. As much as the NCAA downplays the role of RPI in the committee's selection and seeding process, Scott says it can't help but be "subliminally impacted" by the metric, and he's exactly right: Every "nitty gritty" page the NCAA uses, every fact and figure and list of top 50 wins and strength of schedule and noncon SOS and you name it is broken down based on RPI. You can't sit in the selection room and not be affected by RPI. It underpins every consideration the committee makes, whether the committee always knows it or not.

4. Everyone knows this.

Van Pelt ended with a call to media action: "It's on us to be smarter in how we position it. It's on us not to throw up the resumes side-by-side with, 'here's their wins against the top 50.' It's their wins against the top 50 based on the RPI!"

This is the only part with which I disagree. When we do Bubble Watch every week, for example, or when the TV folks throw up said resumes side by side, we're all doing so because we're trying to figure out how the selection committee will decide. The committee plays by stupid, outdated rules, and if we're going to predict what the committee is going to do, we have to be stupid and outdated, too. The Watch has made this clear before.

This week, the NCAA convened its two-day mock selection committee, a chance for media members to learn more about the process, ask questions of the NCAA brass, and experience the unique form of blinding boredom that comes from debating whether or not Colorado State deserves another look. (It's also a chance for media members to live-tweet the proceedings, and thus inflict said blinding boredom on each of their poor, unsuspecting followers.) Each year, the NCAA gets questions from the assorted media. Each year, at least one of these questions concerns the RPI. And each year, the NCAA gives some form of the answer's Jeff Rabjohns tweeted Thursday: "RPI is a historical measurement of factual results of a season, is not predictive, therein lies the value." The NCAA's line is as such: The RPI isn't the only thing we use. It's just a tool. But we think it's a valuable, fact-based tool, and we're going to keep using it. The end.

This is incredibly frustrating! It's frustrating not only because the NCAA's "Process Principles for Selection, Seeding and Bracketing" do not explicitly require the committee to use the RPI. It's also frustrating because most of the metrics discussed above do much the same. The only difference? They do it better.

One day, in a brave new world, the NCAA will do what it should have done years ago. It will chuck the RPI. It will find a far more advanced, reality-based data point (I'm a company man, so I'll propose BPI, but KenPom would do just fine, too) to use as its preferred computer metric. Or maybe not. The point is, until the NCAA decides to join the rest of us in the 21st century, where we all point and laugh at the RPI (and anyone who thinks it's the best numerical metric by which to evaluate a basketball team), we're stuck. In the selection committee's world, it's 1980 forever.

The good news? If you never got to catch the Talking Heads in concert, you're in luck. The bad news? The best sporting event on Earth is being selected and seeded by luddites. Bummer, huh?

Update: Twitter follower Matt Frese makes an excellent point, and one I forgot to address in this post, here: They're never going to chuck RPI for KenPom or Sagarin or BPI because those metrics use Margin of Victory. ... Sure, these other models are more predictive, and just better. But the NCAA is never going to reward blowouts (end rant)."

Matt's right: The NCAA doesn't want to reward blowouts, nor should it. Why encourage teams to run up the score? In the immortal words of Rick James, I think I have the remedy: The BPI accounts for this in its new formula by providing diminishing returns for blowouts: "By capturing blowouts, but not overweighting them, BPI credits the ability of good teams to easily beat poor teams without providing incentive to win by 30 when 20 is a safe margin. By capturing both blowouts and close games in this way, BPI summarizes a team's résumé for the NCAA tournament well." See? Problem solved.
Tuesday, this here blog space discussed a relatively important occurrence in the history of Northwestern basketball: This week, for the first time since 1969, the Wildcats were ranked in the AP top 25. Naturally, the post finished with questions about NU's tournament status. The Wildcats have never qualified for an NCAA tournament, despite years and years of trying, and so that is the major question. Can NU get enough wins in 2010? Will the Wildcats' 10-win nonconference run see them -- and their RPI -- through to the other side of Big Ten play? Is this really the year?

Northwestern fan blog Sippin On Purple took a math whiz's approach to Northwestern's projected RPI and how many wins the Wildcats need in conference to make the tournament. The end result was somewhere around 10 or 11, with 11 and up being the safe side, and 10 or down being rather sketchy.

After reading that post, current Sporting News columnist and blogger and all-around man-about-Internet Dan Shanoff used the G-chat tubes to float me an interesting idea: Will the selection committee be "rooting" for Northwestern? Is Northwestern's bubble more forgiving than any other's? The following is cropped from our conversation:
The Sippin On Purple guys are discounting a huge factor that has nothing to do with number-crunching: the selection committee will do anything they can, within stretched reason, to put NU in the tournament. I could see them getting in with an RPI in the low 60s, ahead of 10 teams who don't get in with RPIs int he 50s and 40s. The committee has a vested interest in putting NU in the tournament if they can. In other words: Northwestern's bubble isn't the same as everyone else's. /rant

If they're in the 70s ... obviously, no chance. And, yes, the 60s makes it a really tough sell. But there won't be a media person on the planet that will begrudge NU being the last team in, leapfrogging over 10 more qualified bubble teams. (Especially since half the media went to Northwestern.)

The committee will be insulated from criticism when it comes to Northwestern. Anyway. Just a thought.

Like I said, it's an interesting theory. Who won't be rooting for Northwestern to get in the NCAA tournament? (Besides, of course, Illinois fans, provided Illinois is on the bubble come tourney time.)

The only problem is that sometimes the selection committee completely discounts media consensus. Sometimes, the committee just doesn't really seem to care. There are such instances every year. Whether it's Arizona getting in over Creighton last year, or "too many" mid-majors getting at-large bids in years past (former CBS analyst Billy Packer famously railed against the latter), there's always at least one big media complaint, whether right or wrong, about the way the field of 65 is seeded. And every year, with nothing more than the occasional glimpse behind the curtain and the occasional post-selection interview to whet our (fans' and the media's) curiosity, the selection committee goes on doing its thing. It doesn't really seem to care much about you and me.

So sure, maybe Northwestern gets the added emotional benefit of the doubt. Or maybe the selection committee just doesn't care about little old romantic underdog Northwestern. Which is more likely?