RPI SOS question

[hussybulldog]

Do you know who cares about RPI/SOS numbers?
Stanford is ahead of UConn in both categories , they face 7 ranked teams this year vs UConn's 8, they have lost one vs UConn's none. I know RPI/SOS is related to the overall record, but does any of these two number signifies anything of any importance? Why do people care about them?

 

[Frodo]

I've written about the RPI before, but I always enjoy another chance to crap on it. Unfortunately, the tournament committee does pay attention to it, though they won't acknowledge how much weight they give it. Just as unfortunately, the RPI is Stop...always has been. Not sure they use the same formula that they used to, but essentially here's how it works; take your team's winning percentage, take the winning percentage of teams that your team played, and take the winning percentage of teams that played the teams your team played. Use the first number as 50% of the final RPI and the latter two as 25% each. What ends up happening, of course, is that it is very heavily weighted by who you play and less by how you did. If you look at the RPI you'll see this clearly since most of the top 20 also have very high SOS ranks. Score differentials also don't count, just wins and losses.

In recent years they've also added something called the Team Power Ranking, which much more closely match, e.g., Sagarin's ratings and are clearly a much better yardstick. I'm not sure how these are calculated, but at least they don't end up with a team like St. Joseph's at #2 after beating a slew of middle of the road teams and losing to LSU. In the interest of full disclosure, they did recently lose to Syracuse and dropped all the way to #6. Anyone who thinks St. Joseph's is the 6th best team in the country, let alone 2nd best, raise your hand...tick...tick...tick...that's what I thought.

 

[stamfordhusky]

I've written about the RPI before, but I always enjoy another chance to crap on it. Unfortunately, the tournament committee does pay attention to it, though they won't acknowledge how much weight they give it. Just as unfortunately, the RPI is bull ...always has been. Not sure they use the same formula that they used to, but essentially here's how it works; take your team's winning percentage, take the winning percentage of teams that your team played, and take the winning percentage of teams that played the teams your team played. Use the first number as 50% of the final RPI and the latter two as 25% each. What ends up happening, of course, is that it is very heavily weighted by who you play and less by how you did. If you look at the RPI you'll see this clearly since most of the top 20 also have very high SOS ranks. Score differentials also don't count, just wins and losses.

In recent years they've also added something called the Team Power Ranking, which much more closely match, e.g., Sagarin's ratings and are clearly a much better yardstick. I'm not sure how these are calculated, but at least they don't end up with a team like St. Joseph's at #2 after beating a slew of middle of the road teams and losing to LSU. In the interest of full disclosure, they did recently lose to Syracuse and dropped all the way to #6. Anyone who thinks St. Joseph's is the 6th best team in the country, let alone 2nd best, raise your hand...tick...tick...tick...that's what I thought.

Actually the RPI uses 25% of your record, 50% of your opponents' W-L pct, and 25% of your opponents' opponents' W-L pct. Your own W-L pct is adjusted to give more weight to road wins and less to home wins etc. The RPI doesn't work well at all until late January, which is why the first official one is not even released by the NCAA until some time in January. No-one should look at it until at least the end of January. By the end of February it generally works pretty well and won't have a lot of big differences from Pomeroy, Sagarin, etc - although at the extremes there may be some issues.

I've never seen anything about the addition of any power rankings. There definitely is not anything like that on the men's side. The NCAA specifically did not want anything like that because it gives teams an incentive to run up the score, use starters the whole game in blowouts, and press the whole game. For that reason, I'd be surprised if they use anything like that in the women's selection process. Of course, RPI and other rankings are only a part of the selection process. Many times teams with lower RPI's are placed ahead of teams with higher RPI's.

BTW, the computer models used for the BCS football selection process, like the RPI, are not allowed to use score differential for the same reasons. Thus the Sagarin model used, for example, is different from the one that shows up in USA Today.

 

[DobbsRover2]

Actually the RPI uses 25% of your record, 50% of your opponents' W-L pct, and 25% of your opponents' opponents' W-L pct. Your own W-L pct is adjusted to give more weight to road wins and less to home wins etc. The RPI doesn't work well at all until late January, which is why the first official one is not even released by the NCAA until some time in January. No-one should look at it until at least the end of January. By the end of February it generally works pretty well and won't have a lot of big differences from Pomeroy, Sagarin, etc - although at the extremes there may be some issues.

I've never seen anything about the addition of any power rankings. There definitely is not anything like that on the men's side. The NCAA specifically did not want anything like that because it gives teams an incentive to run up the score, use starters the whole game in blowouts, and press the whole game. For that reason, I'd be surprised if they use anything like that in the women's selection process. Of course, RPI and other rankings are only a part of the selection process. Many times teams with lower RPI's are placed ahead of teams with higher RPI's.

BTW, the computer models used for the BCS football selection process, like the RPI, are not allowed to use score differential for the same reasons. Thus the Sagarin model used, for example, is different from the one that shows up in USA Today.

That's a bit of a superficial gloss of RPI. No, things don't necessarily even out to being anything normal come February or March. For the top teams it's often just a pileofpoop. For the middle group of teams in the 20-50 range it might be slightly better, but who cares, because those are not the teams that the NCAA tourney revolves around and that the RPI system was developed for. Quite simply, Stanford and Duke will likely be ahead of UConn at the end of the season unless they are kidnapped by disgruntled aliens, and that is not a system that works "pretty well." Check the RPI for that undefeated UConn 2002 team that was widely considered the best team ever at the time, but not in the RPI where they were a distant second to the Vols and their 4 losses. The system is silly, nonsensical, and was clearly developed by demented penguins tripping on some type of bad substance.

Sagarin has a few issues, but is basically pretty close to real world scenarios unless sometimes you're talking about predicted scoring margins for an unreal team like UConn. But Sagarin had UConn beating Cinci by 35, and they won by 33. Not bad.

 

[stamfordhusky]

That's a bit of a superficial gloss of RPI. No, things don't necessarily even out to being anything normal come February or March. For the top teams it's often just a pileofpoop. For the middle group of teams in the 20-50 range it might be slightly better, but who cares, because those are not the teams that the NCAA tourney revolves around and that the RPI system was developed for. Quite simply, Stanford and Duke will likely be ahead of UConn at the end of the season unless they are kidnapped by disgruntled aliens, and that is not a system that works "pretty well." Check the RPI for that undefeated UConn 2002 team that was widely considered the best team ever at the time, but not in the RPI where they were a distant second to the Vols and their 4 losses. The system is silly, nonsensical, and was clearly developed by demented penguins tripping on some type of bad substance.

Sagarin has a few issues, but is basically pretty close to real world scenarios unless sometimes you're talking about predicted scoring margins for an unreal team like UConn. But Sagarin had UConn beating Cinci by 35, and they won by 33. Not bad.

I said at the extremes there might be some issues. But that is not a big deal since the RPI is not used to select the top teams - nor their seeds - anyway.

The NCAA would never use something like Sagarin - nor should they - since it would lead to tons of ill will as teams go out of their way to run up scores. Scoring differential is all-important in those metrics such as Sagarin.

Also let's not forget that the selection committee looks at more than a dozen things, including what each team did in each game, and that someone on the committee has closely studied each league and each team that might deserve a spot. The guidelines for the selection committee mentions the resources available as the committee meets:

Among the resources available to the committee
are complete box scores, game summaries and
notes, pertinent information submitted on a
team’s behalf by its conference, various
computer rankings, injury reports, head-to-head
results, chronological results, Division I results,
non-conference results, home, away and neutral
results, rankings, polls and the NABC regional
advisory committee rankings.

In any event, this conversation is very premature, since everyone, including the NCAA, acknowledges that the RPI doesn't work this early in the season. At the end of February it will be completely different. Definitely not perfect but unquestionably much better.

 

[DobbsRover2]

I said at the extremes there might be some issues. But that is not a big deal since the RPI is not used to select the top teams - nor their seeds - anyway.

The NCAA would never use something like Sagarin - nor should they - since it would lead to tons of ill will as teams go out of their way to run up scores. Scoring differential is all-important in those metrics such as Sagarin.


In any event, this conversation is very premature, since everyone, including the NCAA, acknowledges that the RPI doesn't work this early in the season. At the end of February it will be completely different. Definitely not perfect but unquestionably much better.

The question posed in this thread is what importance is the RPI and why should anyone care about it. As noted, I'm saying that no one should care about it because it is a useless silly system and no one has ever proven that the selection committee real cares a fig about it. Yeah, it's one of many tools, but the the complete idiocy of it in many basic comparisons such as UConn and Stanford and Duke this year (And why is that an extreme? Shouldn't getting the #1 team right be a basic rule for any system?) has proven its minimal value.

Now Sagarin seems to do very nice job overall of assessing teams. As you well know that is just a red herring you're throwing out about runaway scores because:
1. Coaches do not tell their teams to run up scores because "Gee, let's boost our Sagarin rankings." Ill will? Yeah, sure.
2. Sagarin involves a couple of different ratings systems that are factored in, and one of the main ones is basically just a who won - who lost rating. Sagarin uses more nuanced systems some of which look at just the outcome ("In PURE_ELO, only winning and losing matters; the score margin is of no consequence." That ring a bell?) and some of which say that maybe it is important whether the #2 team loses to the #1 team by 22 points instead of 1 point. Then there is that other ridiculous system that says the #2 team is better because they played another team that played another team, and that's what counts because we don't want to hurt any feelings about just how badly teams get beat by. Yes, it does matter how much teams win by, and UConn winning big time at Duke or at the other highly ranked teams' courts rather than eking out a victory is an important rating factor. If there is a good system like Sagarin, then go with it and ignore the scare stories about how teams must be running up the scores for a rating system. Believe me, there are dozens of obscene blow-outs every week and Sagarin seems to deal with them quite well.
3. The NCAA selection committee are not a bunch of imbeciles despite a few questionable placements, and yes they definitely do consider the better tools available to them when making selections rather than relying brain-dead systems. Among the many tools they use are the AP and USA Today rankings, and looking at the current ones, they seem to fit a huge tad more closely to Sagarin than to RPI.
4. Again, no in February the RPI will not be acceptable. As noted, UConn could win the rest of its games by 100 points and it would still finish way behind Stanford and Duke and maybe a few other teams unless they have meltdowns. To me that's not just "not perfect," it's idiotic. But to each their own.

 

[DobbsRover2]

Do you know who cares about RPI/SOS numbers?
Stanford is ahead of UConn in both categories , they face 7 ranked teams this year vs UConn's 8, they have lost one vs UConn's none. I know RPI/SOS is related to the overall record, but does any of these two number signifies anything of any importance? Why do people care about them?

BTW: maybe a change of name could be in order, since although your question is not bad, you do have something about you that smells of the Summitt. First part of your name sounds like a word that Maria uses twice in every sentence whenever UConn is mentioned. Just saying.

 

[Frodo]

The NCAA would never use something like Sagarin - nor should they - since it would lead to tons of ill will as teams go out of their way to run up scores.

Do you really think teams don't run up the scores now? I didn't see the Kentucky-Grambling St. game yesterday, but I will guarantee that Mitchell continued to press all over the court and force turnovers for easy buckets late into the game. He does it all the time against weak opponents. He knows full well that the people who vote in the polls, and the people who will be seeding teams for the tournament in March don't see every game, but they do notice scores when a team consistently runs into triple digits.

 

[KnightBridgeAZ]

What a depressing thread.

Whoever said they do not use Sagarin type ratings is correct, for the reasons indicated. They review the cold numbers in that regard.

They do use the RPI and for the most part, it can be fairly good at predicting seed lines. What it tells the committee, essentially, is how well a team did against how strong of competition. Being #1 means nothing in the ranking - but being in the top 25 vs. the middle 25 - 75 is a very significant measure of strength differential. And yes, it is tweaked to adjust for home / away bias although those adjustments are not released to the general public I believe.

 

[DobbsRover2]

What a depressing thread.

Whoever said they do not use Sagarin type ratings is correct, for the reasons indicated. They review the cold numbers in that regard.

They do use the RPI and for the most part, it can be fairly good at predicting seed lines. What it tells the committee, essentially, is how well a team did against how strong of competition. Being #1 means nothing in the ranking - but being in the top 25 vs. the middle 25 - 75 is a very significant measure of strength differential. And yes, it is tweaked to adjust for home / away bias although those adjustments are not released to the general public I believe.

Well, thank the almighty we have the RPI so it can line up the RPI #4 and #22 rated teams for the NC last year, along with the #2 and #9 teams also for the FF. So with WCBB being so predictable, you have to expect that the top four RPI (totaling 10) or thereabouts will be competing in the FF. Last year it's RPI 37 for the FF ranking sum. Real glad the RPI is around to get everything sorted out fairly good for the selections. And every system is adjusted for home / away that I've ever seen, so the RPI isn't anything special there. I'll take the AP poll over the RPI, which at least was a little better at an FF 27 ranking sum.

As said, RPI is a system that is missing lot of brain cells.