The difficult task of superseding the RPI

[Plebe]

The NCAA Is Modernizing The Way It Picks March Madness Teams

Late last month, NCAA officials met with some of basketball’s most prominent analytics experts to remake the way they select teams for the men’s NCAA tournament. Until now, they’ve used the ratings percentage index (RPI) to help guide their decisions, but that stat has become antiquated as far more advanced ranking systems have been developed. Efforts to replace the RPI, though, raise a lot of tricky questions.

According to multiple people I spoke to who were at the meeting, the NCAA is not interested in generating a completely new metric from scratch. Instead, officials favored using multiple ranking systems to create a composite index that would be a resource on selection Sunday. But as the many controversies around college football’s Bowl Championship Seriesshowed, developing a new rating, even one made up of accepted metrics, is full of twists and turns, roadblocks and landmines. Finding the right formula will require asking deep philosophical questions about what a ranking system should try to achieve — and whether certain statistical compromises are even possible.​

 

[Sonny44]

Why only for the men?

 

[Plebe]

Why only for the men?

Well, it's the leadership in NCAA men's basketball that is seeking out the input on how to modernize the statistical tool. It's safe to assume that whatever they come up with will then be adopted by the WCBB leadership and probably by other sports as well. A similar thing happened with the RPI: developed initially for men's basketball but then adopted by the women's game and other sports.

 

[JoePgh]

Isn't it likely that RPI may be the best index that can be devised if one is working under the constraint that Margin Of Victory may not be factored in?

Of course, Massey and Sagarin can predict game outcomes better than RPI, and produce a ranking of teams which looks more reasonable, because they are free to incorporate MOV in their models. But for very sound and understandable reasons, the NCAA cannot do that. If they did, it would provide a clear incentive for blowouts (the more lopsided, the better for the victorious team). None of us would want to watch the games that would take place under that selection method.

 

[Plebe]

Isn't it likely that RPI may be the best index that can be devised if one is working under the constraint that Margin Of Victory may not be factored in?

The article addresses this somewhat: "Today’s statisticians downplay such an uncompromising approach, however, pointing to such solutions as game control, which measures dominance through a team’s average in-game win probability rather than the final score, and strength of record, which measures how difficult it would be for a generic 'good team' to earn a specific team’s record, given its opponents. Both use a mix of metrics that measure how dominant a team was without explicitly accounting for a team’s margin in a way that coaches could manipulate during garbage time."

 

[Phil]

Isn't it likely that RPI may be the best index that can be devised if one is working under the constraint that Margin Of Victory may not be factored in?

No, it is highly unlikely.

Whenever you see a formula with round number weights like 25%, 50% and 25%, you can be almost certain (with some rare exceptions) that the numbers were pulled out of the air. A better approach would be to do a statistical regression analysis, and solve for the weights that best predict results. It isn't all that hard, but almost certainly wasn't done, so there is some room for improvement, without considering other options, simply by solving for the optimum weights.

It is possible that the optimal weights will not improve the results enough to make it worthwhile, but it ought to be explored.

And that's without considering other ignored factors, such as home versus away, distance, time zones, and other factors.