How Accurate Can The RPI Really Be??? | The Boneyard

How Accurate Can The RPI Really Be???

Status
Not open for further replies.
Something is rotten in Denmark.... (as they say).
 
The emphasis placed on strength of schedule and UConn's weak conference schedule hurt them in the RPI.
 
The RPI scale was proven brain dead back in 2002 when a pre-tourney 4-loss UTenn team that didn't even win its conference championship was rated way way higher than an undefeated UConn team that had also beaten the Vols. Sagarin and Massey are the better ratings services, and of course Massey has UConn rated with the top SOS even before it gets into the two Louisville and the second Rutgers game in a nicely robust AAC conference this year.
 
The RPI is not a power rating. It is not meant to measure the strength of teams, per se. It is meant to measure wins/losses against the quality of competition. If the NBA's Miami Heat played in the MEAC conference, they would have a bad RPI, even though they clearly are better than any college team. This is by design.

Not only does the NCAA not want to be rewarding large margin of victories (which would encourage teams to run the score), but it also has an interest in pushing teams to schedule quality competition. It is good for the game, and perhaps more significantly makes it easier to seed teams. For example, how good is Louisville this year? Hard to tell, since they are 1-1 against the top 25, and 0-0 against the top 10. It makes it hard to seed such a team, and thus the RPI punishes the weaker schedule.
 
Last edited:
.-.
vowelguy said:
As I've said countless times, the RPI is not a power rating. It is not meant to measure the strength of teams, per se. It is meant to measure wins/losses against the quality of competition. If the NBA's Miami Heat played in the MEAC conference, they would have a bad RPI, even though they clearly are better than any college team. This is by design.

Not only does the NCAA not want to be rewarding large margin of victories (which would encourage teams to run the score), but it also has an interest in pushing teams to schedule quality competition. It is good for the game, and perhaps more significantly makes it easier to seed teams. For example, how good is Louisville this year? Hard to tell, since they are 1-1 against the top 25, and 0-0 against the top 10. It makes it hard to seed such a team, and thus the RPI punishes the weaker schedule.
Expand ...
Yes they were fortunate to leave RU with a win though RU wasnt ranked.
 
vowelguy said:
As I've said countless times, the RPI is not a power rating. It is not meant to measure the strength of teams, per se. It is meant to measure wins/losses against the quality of competition. If the NBA's Miami Heat played in the MEAC conference, they would have a bad RPI, even though they clearly are better than any college team. This is by design.

Not only does the NCAA not want to be rewarding large margin of victories (which would encourage teams to run the score), but it also has an interest in pushing teams to schedule quality competition. It is good for the game, and perhaps more significantly makes it easier to seed teams. For example, how good is Louisville this year? Hard to tell, since they are 1-1 against the top 25, and 0-0 against the top 10. It makes it hard to seed such a team, and thus the RPI punishes the weaker schedule.
Expand ...
Um, so you're telling us that since UConn is listed at the #25 schedule in an RPI system that you say is supposed to "measure wins/losses against the quality of competition" that it makes any sense that teams like St. Joes, and St. Johns, and Minnesota and Oregon St. and etc. etc. have a higher level of "quality of competition" this year?

Yes, RPI measures something, and it's something stupid. And yes it is based on measuring teams against its own weird cousin's cousin's cousin form of SOS evaluation, but that has nothing much to do with any sensible "quality of competition" rating system. So, no, it's neither meant to measure the strength of teams or measure the quality of the competition. It is simply brain dead.

As to the old gremlin of teams indeed beating up on each other if they were rewarded for blowing away weak teams, as has been mentioned by many posters ad nauseum on the BY, teams do get crushed by 60 to 100 points quite often and there is no shred of evidence that the situation would change if the RPI system used winning margins and that then elite coaches would start saying, "Gee, we'll look a lot better in the RPI if we pop another 20 points on our 30 point margin." And non-brain-dead systems like Sagarin and Massey handle the point margins very nicely, even if in Sagarin's case the SOS still is computed on a dumb linear and non-relational format unlike Massey.
 
Last edited:
DobbsRover2 said:
Um, so you're telling us that since UConn is listed at the #25 schedule in an RPI system that you say is supposed to "measure wins/losses against the quality of competition" that it makes any sense that teams like St. Joes, and St. Johns, and Minnesota and Oregon St. and etc. etc. have a higher level of "quality of competition" this year?.
Expand ...
Yes, he's told us countless times.

Boy, he got grumpy since some didn't like how he rated Nurse in his combined ratings.
 
DobbsRover2 said:
The RPI scale was proven brain dead back in 2002 when a pre-tourney 4-loss UTenn team that didn't even win its conference championship was rated way way higher than an undefeated UConn team that had also beaten the Vols. Sagarin and Massey are the better ratings services, and of course Massey has UConn rated with the top SOS even before it gets into the two Louisville and the second Rutgers game in a nicely robust AAC conference this year.
Expand ...
What in the world is surprisingly robust about the AAC this year??????????

I track wins and losses and to whom on a spreadsheet. Its a 2 team conference - UConn and Louisville. Rutgers has a slight edge on Memphis, SMU, South Florida and Temple all of whom - when you look at who they have beaten and who they have lost to - just scream undistinguished. And only Rutgers has a decent shot at the dance I'm expecting and even they have work to do.

Cincy, UCF and Houston are, in a word, dreadful.

And while I am someone who finds value in the RPI, I agree that the SOS number is difficult to completely wrap your hands around. UConn has played some very, very good teams and some really awful teams, and not as much meat in-between as the old BE afforded. But top SOS?? Only if you base SOS on weighting the top competition as more important than the dregs. Tell me that no one has played more "strong" teams than UConn, I'm in. But tell me the over all strength of UConn's schedule is tops, between the AAC and the normal couple of cupcakes that every team has OOC and there is no way.
 
DobbsRover2 said:
As to the old gremlin of teams indeed beating up on each other if they were rewarded for blowing away weak teams, as has been mentioned by many posters ad nauseum on the BY, teams do get crushed by 60 to 100 points quite often and there is no shred of evidence that the situation would change if the RPI system used winning margins and that then elite coaches would start saying, "Gee, we'll look a lot better in the RPI if we pop another 20 points on our 30 point margin." And non-brain-dead systems like Sagarin and Massey handle the point margins very nicely, even if in Sagarin's case the SOS still is computed on a dumb linear and non-relational format unlike Massey.
Expand ...



Common sense says the situation would change dramatically for teams that are near the bubble, knowing that a 30-point win is much more valuable than a 15-point win.

Sagarin and Massey are not used for tournament selection, so they obviously have no effect on the behavior of coaches. An RPI using score differential would have a huge effect on the behavior of coaches - unless the coaches are incredibly stupid.
 
.-.
stamfordhusky said:
Common sense says the situation would change dramatically for teams that are near the bubble, knowing that a 30-point win is much more valuable than a 15-point win.

Sagarin and Massey are not used for tournament selection, so they obviously have no effect on the behavior of coaches. An RPI using score differential would have a huge effect on the behavior of coaches - unless the coaches are incredibly stupid.
Expand ...
So what you're worried about is that some teams late in the season will suddenly go against usual conventions and just keep running up the score on a conference mate? And that's a reason to keep a brain damaged rating system the way it is? Wow!

Few points:
  • None of the good rating systems such as Sagarin or Massey reward teams for 50 point wins vs 30 point wins or up a team's rating because they blew away a team by 70. They have margin adjustment factors that minimize the blow-out factor. So your point about running up the score is invalid. But the fact that UConn has a 36 point winning margin and Duke has a 20 point winning margin does get factored into the good systems, and thus Duke is actually rated behind UConn in them instead of ahead of UConn like in the RPI. As I noted, I prefer the Sagarin and Massey methods, but you are free to like RPI instead.
  • As we know, the selection committee just uses the RPI as just one of many general tools for sorting teams into comparable groups, and it isn't even a main tool as they give much more credence to other factors like top victories that they know are not so brain dead. But you're saying that somehow coaches will be all focused on the RPI and cater their game plan to it if the system was changed to include some type of margin-of-victory factor? Man, you got me on that one.
  • So there is this system that rates the nation's #1 team UConn as only #4. Not only that but the team at #1 is not even the other undefeated team and #2 ranked ND but instead Stanford that plays in the worst of the P5 conferences. But this RPI system should not be tinkered with but should be left in its comatose state for what reasons? Because some coaches might run up the score, even if as noted above the system could be set up to minimize they blow-out effects? In case you weren't watching the scores of games this year, there are a helluva lot of blow-outs even with the RPI not caring. UConn just won a game by 61, but maybe you're thinking that except for our cretinous RPI system the margin might have been 70?
There is simply no reason for a selection committee to be using a tool that places UConn at #4, or maybe even #5 or #6 by season's end. You may be happy with that, I'm not.
 
Last edited:
vowelguy said:
As I've said countless times, the RPI is not a power rating. It is not meant to measure the strength of teams, per se. It is meant to measure wins/losses against the quality of competition. If the NBA's Miami Heat played in the MEAC conference, they would have a bad RPI, even though they clearly are better than any college team. This is by design.

Not only does the NCAA not want to be rewarding large margin of victories (which would encourage teams to run the score), but it also has an interest in pushing teams to schedule quality competition. It is good for the game, and perhaps more significantly makes it easier to seed teams. For example, how good is Louisville this year? Hard to tell, since they are 1-1 against the top 25, and 0-0 against the top 10. It makes it hard to seed such a team, and thus the RPI punishes the weaker schedule.
Expand ...


That's quite interesting.

I've always known that the RPI excluded MOV because they didn't want to run up the score, which puts them at a severe disadvantage (from an information theory point of view, they are throwing away useful and relevant information).

It means their goal is not simply to be a good metric, but to change behavior.

However, I hadn't considered the SOS implications, that again, they might give up the statistically optimal weighting in favor of a weighting which encourages certain behavior (scheduling tougher competition.)

It actually make s me feel a little better about the measure, because I've usually only viewed it as a measuring tool, at which it is quite deficient.

I said "a little" deliberately, because it is still easy to game the system with your scheduling, and some schools seem to have done so.
 
I don't like the RPI because it punishes teams for beating bad teams. Beating a bad team really says nothing about a good team, especially in WCBB. They should only use your record against good teams. Volume of good teams beaten should carry much more weight than the fact that a team played some bad teams. A bad loss also obviously says something. I just don't see why UConn drubbing some bottom feeder should matter at all.
 
KnightBridgeAZ said:
What in the world is surprisingly robust about the AAC this year??????????
Expand ...
Listen, we know Rutgers is running off to the queen of all conferences next year to become part of an august group that will have no top teams and continually underperform in tourney play. That's not a great reason for the team's fans to try to foul the nest they leave behind.

As noted, last year on the BY there was little but expressions of horror about what the AAC would be like this year. There was mention of a few conferences like the WCC that would probably be better than the AAC. Didn't turn out that way, and in fact in one of Sagarin's ratings the AAC is better than the PAC. That seems like a good reason to say the AAC is surprisingly robust. And I'm glad to say Rutgers is back from the dead and is one of the reasons that the AAC has been unusually chipper.

When Rutgers came over to the BEast in the mid-1990s, it was bad -- maybe not Houston bad but I would take this year's Cinci with a win against OSU over a 1996-97 Rutgers team. But what happened? Rutgers went on to become pretty good and gave UConn a big challenge for many years. That was a story that happened for a number of BEast teams that originally were weak sisters.

So how can the AAC's robusto brew continue after cherished Rutgers and Lville run off? Well, instead of having just a huge bunch of sad-sack mid-1990s Rutgers type teams filling in behind UConn, the AAC has last year's tourney team USF, and teams like Temple and SMU that are rated ahead of at least some B1G teams like Wisconsin. CUSA teams East Carolina and Tulane are also winning a lot of games this year and should rate much better when they move to the AAC. Memphis at least beat Rutgers, and even a healthy Cinci could make some noise, as it did this year knocking off another traditionally overpuffed B1G team OSU.

How long before this AAC collection can surpass the rust belt B1G once they start banging heads against the Huskies more? Maybe a few years. But just as the B1G fans used to deride the Big East back in the mid 1990s, those who say that the AAC is just destined to be weak while the B1G will somehow be pulled out of its morass by MD and Rutgers are likely to be disappointed. Sheesh, even the PAC gets more attention these days than the B1G.
 
DobbsRover2 said:
Listen, we know Rutgers is running off to the queen of all conferences next year to become part of an august group that will have no top teams and continually underperform in tourney play. That's not a great reason for the team's fans to try to foul the nest they leave behind.

How long before this AAC collection can surpass the rust belt B1G once they start banging heads against the Huskies more? Maybe a few years. But just as the B1G fans used to deride the Big East back in the mid 1990s, those who say that the AAC is just destined to be weak while the B1G will somehow be pulled out of its morass by MD and Rutgers are likely to be disappointed. Sheesh, even the PAC gets more attention these days than the B1G.
Expand ...
I'm not fouling the nest. I hate the style of play that has been typical of the B1G WBB and led to somewhat decreasing relevance on the stage of WBB. (Not fond of the PAC style either) And I never thought that Rutgers or Maryland are going to save the league. It isn't about WBB - in fact, WBB is completely irrelevant, much as it hurts me to say it, to Rutgers being in the B1G.

I'm sorry, but objectivly, the AAC - including Rutgers - is mediocre this year. I never claimed it was going to be awful like conferences you name; I consider it to be somewhere in the #6, #7, or #8 conference. You obviously don't really realize how bad some conferences are. All the "mid" AAC teams can beat a B1G, or even ACC opponent, or SEC or PAC or whatever. There are conferences that 90% of members couldn't dream of doing that. Are they going to beat a lot of them - not until they improve, and like the Big East, they will undoubtedly improve. How much is the question.

Surpass the B1G as a WBB conference? Certainly possible. I don't personally think it likely, but the good thing about RU being in the B1G is revenue. RU loses money at athletics (and lord knows does plenty of things on the cheap because of it). The guaranteed money - primarily from football bowl tie-ins and the Big Ten network - are vital. It is unlikely to ever be worse than the 5th conference in FB or MBB, and, I personally think WBB as well.

Most importantly for RU was the academic tie-ins of the B1G, because Rutgers is undergoing a major program of redefining its goals and positioning its academic side in the changing face of academics. The lengthy report just issues points out the importance of the B1G's academic organization to the goal of furthering Rutgers academic aims.
 
And on a separate note about Rutgers in the mid-'90s: Coach Stringer has said often that she would not have come to Rutgers if they were not joining the Big East and the conference that had UConn in it. So in that regard, you are correct that UConn is responsible for RU's improvement.

However, Coach Grentz, who left when we joined the BE, did leave the cupboard bare. The talent level was decent, but CVS once made the comment that - after the first practice - she went up to her office and cried. It took 2 losing seasons for RU to make the sweet 16 in CVS's 3rd year.

As to Tulane and E. Carolina, I don't follow them; traditionally, I have never thought much of Tulane, but have periodically heard E. Carolina noted for some success.
 
.-.
I think Jimmy Fallon has a rating system that is sometimes as accurate as the RPI. He takes a bunch of Labradoodles and Germanoodles with numbers on them and gives them basketballs with team names on them and however they pair up is the weekly ranking. He also uses them to predict winners of NCAA, NFL, NBA and MLB games, series and bowl games.
 
KnightBridgeAZ said:
I'm sorry, but objectivly, the AAC - including Rutgers - is mediocre this year. I never claimed it was going to be awful like conferences you name; I consider it to be somewhere in the #6, #7, or #8 conference. You obviously don't really realize how bad some conferences are. All the "mid" AAC teams can beat a B1G, or even ACC opponent, or SEC or PAC or whatever.
Expand ...
I think the point about the AAC is that it has potential, and the experience of playing against UConn has made a lot of weak teams up their games a lot, occasionally even winning an NC or at least playing in an NC game. You're right, there are no superconferences in WCBB. Pick a P5 conference and you can reel off a list of demerits about its situation. And I think that's precisely the type of situation that a scrappy up-and-coming AAC headed by a powerful standard bearer can do reasonably well in. Having 6 of 11 of next year's AAC teams in the Sagarin top 100 is a start for developing from, and it's certainly way better than having just UConn and 10 teams in the 100-200 range, which was being blubbered about last year around this time.

No, the money situation for most AAC schools is on the bleak side, but that doesn't seem to matter much for WCBB teams, which don't exactly get the courts coated in gold for them. I would agree that UConn football is now at a huge competitive disadvantage (as it always has been) against even the most mediocre SEC school, but since when has the cable money ever made and Alabama or Auburn a top WCBB team in recent times? Auburn's time as a WCBB superpower was long before the SEC got the money bloats. And Rutgers will get the money in the B1G, but gearing up to play at Purdue or PSU's level in its top games has got to be a future downer after having once had UConn, ND, and Louisville in those top spots.
 
If the RPI were the sole criteria used, then yes that would be a problem. But it is not. It is not designed to serve as the be-all, end-all in evaluating teams. Like any ranking system it has strengths and weaknesses. So long as it isn't over-intepreted, it is a fine measure.

Also, just because it doesn't rank well at the top, doesn't mean that it doesnt do a good job in the middle of the ranks.

Consider someone's GPA in college. Is it a perfect match for how someone will perform in the workforce? No. But it does convey some information about how a person performed against the coursework vs others. And it is one piece of information that employers look at when hiring.
 
vowelguy said:
If the RPI were the sole criteria used, then yes that would be a problem. But it is not. It is not designed to serve as the be-all, end-all in evaluating teams. Like any ranking system it has strengths and weaknesses. So long as it isn't over-intepreted, it is a fine measure.

Also, just because it doesn't rank well at the top, doesn't mean that it doesnt do a good job in the middle of the ranks.

Consider someone's GPA in college. Is it a perfect match for how someone will perform in the workforce? No. But it does convey some information about how a person performed against the coursework vs others. And it is one piece of information that employers look at when hiring.
Expand ...
Again VG, basically agree on all points except one. We know that the RPI often does demented a job for the top teams, which is indisputable. But then we need to accept that RPI does "a good job" for the middle-ranked teams. How do we prove that statement? Right now RPI has WVU ranked 21st and Sagarin has the team at 11th. MD is 20th in RPI and 7th in Sagarin. St. John's is 11th in RPI and 45th in Sagarin. St. Joe's is 15th in RPI and 25th in Sags. Vanderbilt is 14th in RPI and 25th in Sags. And there are many more huge discrepancies in the way a team is rated in the two systems.

I'm not necessarily saying that Sagarin is the better judge in all cases and RPI is wrong, but seeing how the top teams are rated, I give much more credence to Sagarin on the middle-level teams too. So what proof do we have that the RPI is good for anything other than as a punching bag for jokes?

Quite simply, it is a stupid system that should be replaced by one that is not so obviously flawed and that has some credibility. The selection committee should not have to say, "We're looking at the RPI rankings, but we're just ignoring all the figures at the top." There is no reason it should be used at all except by nostalgic UTenn fans who want to "prove" they had the toughest SOS for many years and many years ago.
 
DobbsRover2 said:
I think the point about the AAC is that it has potential, and the experience of playing against UConn has made a lot of weak teams up their games a lot, occasionally even winning an NC or at least playing in an NC game. You're right, there are no superconferences in WCBB. Pick a P5 conference and you can reel off a list of demerits about its situation. And I think that's precisely the type of situation that a scrappy up-and-coming AAC headed by a powerful standard bearer can do reasonably well in. Having 6 of 11 of next year's AAC teams in the Sagarin top 100 is a start for developing from, and it's certainly way better than having just UConn and 10 teams in the 100-200 range, which was being blubbered about last year around this time.

No, the money situation for most AAC schools is on the bleak side, but that doesn't seem to matter much for WCBB teams, which don't exactly get the courts coated in gold for them. I would agree that UConn football is now at a huge competitive disadvantage (as it always has been) against even the most mediocre SEC school, but since when has the cable money ever made and Alabama or Auburn a top WCBB team in recent times? Auburn's time as a WCBB superpower was long before the SEC got the money bloats. And Rutgers will get the money in the B1G, but gearing up to play at Purdue or PSU's level in its top games has got to be a future downer after having once had UConn, ND, and Louisville in those top spots.
Expand ...
I have read your comments about next year's AAC on several threads and they always mention last year's blubbering about this year's AAC. I'm not sure why that is so relevant. Almost all of your answer boils down to UConn will pull these WBB teams up. Seems to me that these teams (meaning their administrations and ADs) have to want to be pulled up. UConn pulled it self up when it got a great coach. Other great coaches followed in the league. Are there other great coaches now in the AAC? Are there fans at these schools who care enough about WCBB to support what will be a hard journey to make AAC a major conference? Are there enough top recruits who will go to these other teams just because they play UConn twice a year? And when Geno retires, will UConn stay as strong as it is now? I guess I think it will be much harder to make the AAC a major conference than you do. I hope you're right.
 
huskybill said:
I have read your comments about next year's AAC on several threads and they always mention last year's blubbering about this year's AAC. I'm not sure why that is so relevant. Almost all of your answer boils down to UConn will pull these WBB teams up. Seems to me that these teams (meaning their administrations and ADs) have to want to be pulled up. UConn pulled it self up when it got a great coach. Other great coaches followed in the league. Are there other great coaches now in the AAC? Are there fans at these schools who care enough about WCBB to support what will be a hard journey to make AAC a major conference? Are there enough top recruits who will go to these other teams just because they play UConn twice a year? And when Geno retires, will UConn stay as strong as it is now? I guess I think it will be much harder to make the AAC a major conference than you do. I hope you're right.
Expand ...
The answer to all your questions is yes.

In any conference there is a sorting out of the schools about priorities. Not all of the teams in the AAC will decide that WCBB is a showcase sport for them, just like in the BEast there were teams like Rutgers, ND, Louisville, DePaul and Villanova that put relatively considerable resources into the women's teams, and there were schools like Providence and Seton Hall that didn't give a fig and were happy to be doormats. Whether it will be USF or Temple or SMU or East Carolina or Navy or other teams that decide to leap on the opportunity, there will be teams raising their game and moving into slots opened up by Louisville and Rutgers' departures.

OTOH, if UConn did get a B1G bid, then all that is moot, and the Huskies could help to shake a lot of rust off those midwestern teams.
 
.-.
I've mentioned this before, but if the main rationale for ignoring MOV is to keep teams from running up the score, they could accomplished that with a censored MOV; that is measure the actual margin, but any margin over some predefined limit, say 20, is treated as if it were 20. Then if a team is up by 20, there is zero incentive, for ranking purposes, to run the score up. My guess is that 95% of games are within that limit, so almost no information would be lost, and the committee could eat their cake and have it, too.
 
I think any SOS or RPI system that does not have a sliding scale for competition in WCBB is not going to be terribly good. The top ten teams any year should never lose to a team ranked 50 or higher so to award a better 'score' for beating team 51 (by any margin) than beating team #251 (by any margin) is just silly. And I would refine it even more - for a top ten team to lose to a team ranked 35 - 50 is terrible whether the ranking of tat team is 36 or 49. And to lose to a team ranked 25 - 34 is pretty bad whether the team is ranked 26 or 34.
AND for a team ranked 40th to have a win over a team ranked #15 or better is great whether that team is #14 or #3, and to beat a team ranked 16 - 25 is very good whether they are #16 or 24.
So ... pooling those teams ranked far above or below a teams ranking should happen while keeping specific values for the teams within say 20+/- ranking spots. I believe Massey is doing something like this with their rankings though I am not sure. But the idea that if Uconn scheduled teams ranked from 75 - 100 in place of the teams that they have played ranked above 125 would actually improve their 'Strength of Schedule' by a significant amount is ludicrous - against any of those teams Uconn can pretty much chose whatever their point spread will be. But that is what the linear SOS does - play the #2 team in the country and the #198 team in the country and it is the same as playing the #99 and #101 team in the country. For the team ranked #100 you might be able to argue that the expectation is the same but the reality is they have a fair chance of winning or losing both games in scenario 2 and an almost 100% chance of ending at 1-1 in scenario 1, for the team ranked #1 there is an almost 100% chance of 2-0 in scenario 2, and a fair chance of 1-1 in scenario 1.
 
Massey does use a system that has relative comparisons so that a top team playing a top team gets more weight in the team's score than when they play bottom teams, and likewise a bottom team's score is more weighted toward how it does against bottom teams and does not get a big boost for losing by 60 to a top team.

Sagarin uses much more linear rating factors, but it has a couple of different scales and two of them use MOV in some way (the formula is a secret), and the composite score is pretty accurate for predicting game margins.

RPI can give you maybe a 50% chance of predicting which of two monkeys will be the first to hit the dart board.
 
Status
Not open for further replies.

Forum statistics

Threads
168,515
Messages
4,579,929
Members
10,489
Latest member
smAAAll


Top Bottom