RIP RPI

[Ewekon]

very interested to see how this turns out.

i had made a few attempts at developing a machine learning model based on team stats and kenpom to build my brackets in prior years.

the main issue i foresee with this is machine learning methods (neural networks in particular) are usually a black box in that it can be hard to explain why the model made a specific decision. so when this new model leaves your team out of the tournament it may not give you a clear reason why.

and for those that are always looking for the anti-uconn angle in ncaa decision making, perhaps they trained the model with a bias against uconn, to ensure lower rankings in the future.

Your first few paragraphs made you look smart. The last confirmed brilliance.

 

[willie99]

Gamblers have been working on this for generations, with no luck. Math just doesn't provide a solution, no matter how many variables they throw in the mix

It should be about the eye test and a little common sense, but it will still be about what schools or conferences are represented on the committee AND spinning the numbers they use

 

[auror]

Gamblers have been working on this for generations, with no luck. Math just doesn't provide a solution, no matter how many variables they throw in the mix

It should be about the eye test and a little common sense, but it will still be about what schools or conferences are represented on the committee AND spinning the numbers they use

You're right in that the sample sizes are too small and variance in the game too large to be perfect.

But the Eye test is much more flawed than something like KenPom (or hopefully NET), just due to the inputs. Are you going to watch every possession of every team in the entire country? Even just the bubble is probably 80 teams * 34 games * 2 hours = 225 days worth of footage without sleeping. Further, are you going to be able to apply context of relative opponent strength in real time? Are you further able to prevent confirmation and other biases?

There's a reason successful gamblers have had "systems" without relying on the eye test.

 

[Paesano]

This is not perfect but it is progress. The RPI was awful.

 

[Scrutineer]

NCAA won’t apply NET metric to previous yrs for comparison

Now THAT is ridiculous. No need to see if this is a quality metric via back testing. Just trust us...



Ahhhhhh there's the NCAA I know and love. Passing the buck like an old pro.

The wording, "won't apply the NET metric to previous years for comparison" implies they didn't apply it.

Whereas my cynical mind tells me they probably did back test it, but won't allow the results to be seen and used for comparison because it would expose the fact that there's bias built into it.

 

[UConnNick]

Anything Mark Emmert is involved with cannot possibly be an improvement.

 

[weyuo]

" The new ranking system was approved in late July after months of consultation with the Division I Men’s Basketball Committee, the National Association of Basketball Coaches, top basketball analytics experts and Google Cloud Professional Services."

How the #$bleep%$ did Google sleaze it's way into how the NCAA determines it's National Championship bracket?

 

[auror]

" The new ranking system was approved in late July after months of consultation with the Division I Men’s Basketball Committee, the National Association of Basketball Coaches, top basketball analytics experts and Google Cloud Professional Services."

How the #$bleep%$ did Google sleaze it's way into how the NCAA determines it's National Championship bracket?

Google Cloud Named Official Cloud of the NCAA

I remember making fun of this at the time.

 

[nelsonmuntz]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

 

[pepband99]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

Just as many mid majors game the RPI to their advantage in scheduling - it's a 2-way street. I don't mind a little bit of mystery here, mostly because i would be shocked if this doesn't closely follow something like Pomeroy, with some added factors like the points cap. It won't be that hard to figure out.

 

[Purple Stein]

For all its faults, the RPI was simple and teams knew how to build their schedules around it.

NET is a black box that teams won't know how to schedule, and will likely heavily favor the P5 + Big East. This is a big step backwards.

The real question is whether it’s adaptive/frequently tweaked. We want to read bias into the system, and time will tell whether we’re justified!

 

[dennismenace]

Kitty: We are waiting.

 

[auror]

The real question is whether it’s adaptive/frequently tweaked. We want to read bias into the system, and time will tell whether we’re justified!

 

[QDOG5]

Just as many mid majors game the RPI to their advantage in scheduling - it's a 2-way street. I don't mind a little bit of mystery here, mostly because i would be shocked if this doesn't closely follow something like Pomeroy, with some added factors like the points cap. It won't be that hard to figure out.

Just wondering how the mid majors gamed the RPI?

 

[CL82]

Kitty: We are waiting.

Conspiracy Kitty says: Google is a responsible corporate citizen and a fine, fine company. I support them in whatever the do.



[Whispers] Shhh - they are everywhere. Why do you think I wear the foil?

 

[raf46]

This could mean that coaches will tend to keep their starters in longer just to ensure a 10+ point victory. If you think back to the National Championship game in 2004, its crazy to think the final margin was 9 points. I think at some points we were up by almost 30.

yes we were up by 30. maybe a little more!

 

[auror]

Just wondering how the mid majors gamed the RPI?

Famously the Missouri Valley in 2006.

The have-nots play the RPI game, too. The Missouri Valley Conference famously instituted a scheduling mandate a decade ago to improve its overall RPI and, with it, the chances of receiving at-large berths in the NCAA Tournament (and the accompanying seven-figure payouts). Schools risked financial penalties if they didn’t comply.

The MVC rose from a one-bid league to receiving four in 2006, two of which reached the Sweet 16. The scheduling mandates were removed for unrelated reasons, and the MVC became a one-bid league again. But 75 percent of the conference’s revenue is generated from NCAA Tournament payouts, and those payouts are projected to drop from $5 million to $2 million. The scheduling mandates are back.

Post article from 2006

Elgin decided to make a change in early 2000 after seeing the Valley's nonconference strength-of-schedule ranking drop from eighth to 25th. The conference decided to withhold an annual $50,000 NCAA tournament distribution from programs that did not play a nonconference schedule consisting of opponents with a three-year average RPI of 149 or better. Two teams, Elgin said, did not make the cut. But Elgin ditched the policy after two years, and now the league office works in conjunction with teams to assemble nonconference schedules.

Consider the top six teams in the Valley and the ACC, all potential NCAA tournament teams. The ACC group played 43 games against top 50 competition, including 13 opponents outside the ACC. The Valley group played 68 games against top 50 competition, but only eight opponents outside the Valley. In games against top 50 teams outside their conference, the Valley group is 4-4 and the ACC group is 9-4.

 

[Chin Diesel]

So, instead of wanking over playing opponents with low RPI's, we're back to complaining that our opponents just suck?

 

[QDOG5]

That's some good stuff @auror. I graduated from Bradley and didn't know the MVC got 4 teams in the tourney in 05-06. Bradley had a surprise run to the sweet 16 behind 7 footer Patrick O'Bryant. He's near the top of the list of guys who should have stayed in college. Never sniffed the NBA. Eustachy says he got the idea on how to beat the RPI from Cal (surprising) but the articles don't flesh out the methodology. From what I gather the entire conference schedules tough ooc away games before the conference season and hopefully a team or two are lucky to get a win. Then they trade wins and losses in conference after that and the top tier of the conference keep their high RPI. Am I on the right path with my analysis. Also, those articles are quite dated and I'm guessing the major conferences figured out how to thwart them by not playing them early in the season.

 

[auror]

That's some good stuff @auror. I graduated from Bradley and didn't know the MVC got 4 teams in the tourney in 05-06. Bradley had a surprise run to the sweet 16 behind 7 footer Patrick O'Bryant. He's near the top of the list of guys who should have stayed in college. Never sniffed the NBA. Eustachy says he got the idea on how to beat the RPI from Cal (surprising) but the articles don't flesh out the methodology. From what I gather the entire conference schedules tough ooc away games before the conference season and hopefully a team or two are lucky to get a win. Then they trade wins and losses in conference after that and the top tier of the conference keep their high RPI. Am I on the right path with my analysis. Also, those articles are quite dated and I'm guessing the major conferences figured out how to thwart them by not playing them early in the season.

Yeah the important thing was to schedule tough road games, win a couple if possible, avoid scheduling the real bottom of the barrel teams, try to schedule teams that will get a lot of wins but won't necessarily but good enough to beat you, and then win ALL of your home games. And then in conference hope as many home teams win. The RPI was very screwed up with home/road weighting.

But the vortex thing is kinda key for conference-wide system hacking. You need a couple actual good teams in your conference to beat other good teams, and then you need your other teams to schedule so they don't lose home games to mediocre/bad teams in nonconf and then have a couple of your above average teams beat the actual good teams which makes them then look like good teams themselves. It needs a bit of a perfect storm to work, but then it's self-sustaining.

Luke Winn: Scheduleball: Colorado State, Pitt exploit weaknesses of RPI

 

[QDOG5]

Yeah the important thing was to schedule tough road games, win a couple if possible, avoid scheduling the real bottom of the barrel teams, try to schedule teams that will get a lot of wins but won't necessarily but good enough to beat you, and then win ALL of your home games. And then in conference hope as many home teams win. The RPI was very screwed up with home/road weighting.

But the vortex thing is kinda key for conference-wide system hacking. You need a couple actual good teams in your conference to beat other good teams, and then you need your other teams to schedule so they don't lose home games to mediocre/bad teams in nonconf and then have a couple of your above average teams beat the actual good teams which makes them then look like good teams themselves. It needs a bit of a perfect storm to work, but then it's self-sustaining.

Luke Winn: Scheduleball: Colorado State, Pitt exploit weaknesses of RPI

Another good read. Back in the day guys I worked with some guys who ran a pool where the object was for your teams to accumulate as many D1 wins as possible. After the draft I looked at the rosters. There were some blue blood first rounders but there were more Texas Pan American, Nicholls St, and Louisiana-Lafayette type schools in the first round. In the real world you recognize those good low tier D1 teams and schedule them at home. Seems fairly simple. I'm surprised more teams didn't do it. Is this new system supposed to mitigate RPI? Btw, I'm not a fan of using margin of victory as any part of the new metric but I've found when picking NCAA brackets MOV is a pretty good determinant of a teams success. Teams with high MOV rarely get knocked out on the first weekend whereas teams that grind out low scoring wins like back in the day Pitt and Cincy had a tougher time in the first weekend. It doesn't always work (UConns last two titles are proof of that) but many times it does(UConns first two titles, UNC under Roy Williams). Kansas and Memphis were one/two in MOV the year they played in the title game.

 

[auror]

Another good read. Back in the day guys I worked with some guys who ran a pool where the object was for your teams to accumulate as many D1 wins as possible. After the draft I looked at the rosters. There were some blue blood first rounders but there were more Texas Pan American, Nicholls St, and Louisiana-Lafayette type schools in the first round. In the real world you recognize those good low tier D1 teams and schedule them at home. Seems fairly simple. I'm surprised more teams didn't do it. Is this new system supposed to mitigate RPI? Btw, I'm not a fan of using margin of victory as any part of the new metric but I've found when picking NCAA brackets MOV is a pretty good determinant of a teams success. Teams with high MOV rarely get knocked out on the first weekend whereas teams that grind out low scoring wins like back in the day Pitt and Cincy had a tougher time in the first weekend. It doesn't always work (UConns last two titles are proof of that) but many times it does(UConns first two titles, UNC under Roy Williams). Kansas and Memphis were one/two in MOV the year they played in the title game.

The new metric completely replaces RPI. MOV is only a piece of the new metric ("NET") and it seems like the "resume" portion is more important, so winning games against good teams (or teams that get a lot of wins) will still be a key to way to boost your ranking in the new NET ranking..

 

[nelsonmuntz]

Yeah the important thing was to schedule tough road games, win a couple if possible, avoid scheduling the real bottom of the barrel teams, try to schedule teams that will get a lot of wins but won't necessarily but good enough to beat you, and then win ALL of your home games. And then in conference hope as many home teams win. The RPI was very screwed up with home/road weighting.

But the vortex thing is kinda key for conference-wide system hacking. You need a couple actual good teams in your conference to beat other good teams, and then you need your other teams to schedule so they don't lose home games to mediocre/bad teams in nonconf and then have a couple of your above average teams beat the actual good teams which makes them then look like good teams themselves. It needs a bit of a perfect storm to work, but then it's self-sustaining.

Luke Winn: Scheduleball: Colorado State, Pitt exploit weaknesses of RPI

In that article, the argument for ripping the RPI up is that Colorado State was the one team to beat the system in 5 years, and they got what, an 11 seed? I also suspect that Colorado State's RPI SOS vs. KenPom SOS gap was 90% the result of dumb luck, and not an evil plot the Rams' basketball coach hatched because he had perfect knowledge that Denver would be a Top 100 team that season.

Including MOV rewards teams for scheduling weaker opponents and blowing them out. Beating a NEC team by 40 is not the equivalent of beating an MVC team by 5, because the NEC team was never a threat to win that game, while the MVC team was. John Thompson Sr. would have approved of the NET formula.

 

[pepband99]

In that article, the argument for ripping the RPI up is that Colorado State was the one team to beat the system in 5 years, and they got what, an 11 seed? I also suspect that Colorado State's RPI SOS vs. KenPom SOS gap was 90% the result of dumb luck, and not an evil plot the Rams' basketball coach hatched because he had perfect knowledge that Denver would be a Top 100 team that season.

Including MOV rewards teams for scheduling weaker opponents and blowing them out. Beating a NEC team by 40 is not the equivalent of beating an MVC team by 5, because the NEC team was never a threat to win that game, while the MVC team was. John Thompson Sr. would have approved of the NET formula.

No, it compares like-for-like. There is no way the average NEC team is rated as highly as the average MVC team.

The point is comparing two games with similar ratings, not the scenario you suggest. If they weight it right, it works.

 

[willie99]

You're right in that the sample sizes are too small and variance in the game too large to be perfect.

But the Eye test is much more flawed than something like KenPom (or hopefully NET), just due to the inputs. Are you going to watch every possession of every team in the entire country? Even just the bubble is probably 80 teams * 34 games * 2 hours = 225 days worth of footage without sleeping. Further, are you going to be able to apply context of relative opponent strength in real time? Are you further able to prevent confirmation and other biases?

There's a reason successful gamblers have had "systems" without relying on the eye test.

Use Ken Pom to bet games (or just predict ATS results), let me know how that works out for you

And can you name a gambler that's designed a successful system? Something that works for some extended period of time? year after year?

I mean the best system I know of is to bet on the Patriots every week, always a winning season




School presidents and AD's have no idea what's happening on the court, and neither the RPI nor it's replacement is going to fix that

But their presence does improve the chances of their school and their conferences 10 fold

Coach K mastered beating the RPI
1) beat up on schools ranked 150, teams you're only favored to beat by 20 points
2) don't play schools ranked 250, teams you'd be favored to beat by 30 points
3) schedule in-state schools at some large arena near them and call it a neutral court or road game. A secondary benefit is earned if it's an ACC Tournament or NCAA Tournament site.

baddaf'nbing you just owned the RPI

 

[willie99]

The trick to beating the RPI (or any mathematical calculation) is to schedule stronger cupcakes

Team A plays # 1 and # 291
Team B plays two cupcakes, teams 130 and 140

Mathematics will tell us Team B played the tougher schedule, when Team A was the only team to play somebody that could actually beat you at home

It's a simplification of the process for presentation purposes, but that's very much how it works

 

[willie99]

Coach K mastered beating the RPI
1) beat up on schools ranked 150, teams you're only favored to beat by 20 points
2) don't play schools ranked 250, teams you'd be favored to beat by 30 points
3) schedule in-state schools at some large arena near them and call it a neutral court or road game. A secondary benefit is earned if it's an ACC Tournament or NCAA Tournament site.

baddaf'nbing you just owned the RPI

and then it gets even better, because K owned the RPI, every team he plays has a stronger RPI. Then the whole conferences RPI gets stronger, then K's RPI gets stronger still because his opponents' RPI and his conference's RPI are stronger

That's called a self fulfilling prophecy

 

[nelsonmuntz]

The trick to beating the RPI (or any mathematical calculation) is to schedule stronger cupcakes

Team A plays # 1 and # 291
Team B plays two cupcakes, teams 130 and 140

Mathematics will tell us Team B played the tougher schedule, when Team A was the only team to play somebody that could actually beat you at home

It's a simplification of the process for presentation purposes, but that's very much how it works

The best thing to do is not play #291. The RPI encouraged bubble teams to play 1-2 quality midmajors on the road. The result was just enough upsets to keep November and December interesting, and a system that weeded mediocre major conference teams out. The system worked.

I hope this isn't just another tool to get more 18-14 Big 10 and ACC teams in the tournament.

 

[Excalibur]

What is the goal of selecting the teams for the tournament? Is it to reward the "best teams", or the teams that had the best seasons? I think, for the in/out question, it should be based on the latter. The new index seems to concentrate on the former, which I don't like and which will lead to more P5 teams in the tournament.

 

[CTBasketball]

I just think about how many undeserved bids Syracuse will get now...