Program rankings, an analytical approach (long post)

[Tk79]

So how does our program stack up relative to the other elite ones? I posted an analysis on this subject several years ago. I was going to post an update back in January as a dose of perspective when many of us (myself included) were down in the dumps on Uconn’s performance, but I was too lazy. Then we went on a run in March and we all got excited again. My inner voice keeps telling me to wait until after next weekend to post this, but I can’t get any work done and I’m tired of reading all the Boneyard threads for the third time, so I decided to make my own.

The question is simple – how do the top programs compare in terms of performance? The answer is not so simple because it is inherently a subjective topic and the criteria/definition of “performance” are not universal. For example, how much weight should national championships carry? What about final 4s, conference titles, wins? The timeframe is also a subjective input into any analysis like this. Is it a 10 year analysis? 20 years? 25? Or does even longer term historical success count just as much (the UCLA argument)?

There is no right answer here, which is why it’s a fun debate. But whatever argument you use, it must be based on facts. I did some research to collect the relevant facts, put them into a tool that allows you to choose your own criteria of performance, then math takes over and spits out the rankings of each program.

So here’s how it works. I chose a peer set of 12 teams to compare. Besides Uconn, I included the following programs: UNC, Dook, Kentucky, UCLA, Kansas, Indiana, Mich St, Arizona, Florida, Louisville. I also included Syracuse for sh@*s and giggles, but unfortunately for them regular season t-shirts didn’t qualify as one of my performance criteria.

As for the metrics, I chose (arbitrarily) 11 of them. I don’t think they are all equally important (more on this later) but I wanted a flexible tool to accommodate differing opinions. Metrics included are

1. # of National titles: the ultimate measure of success
2. # of final 4s: another commonly used indicator of “elite” seasons
3. # of elite 8s
4. # of Sweet 16s: rewards teams that are consistently solid, if not spectacular, in the tournament
5. # of NCAA appearances: I don’t like this one for elite programs, more useful for lesser programs
6. # of Tourney wins: an aggregate rather than seasonal measure
7. # of Conference tourney titles: need something to balance out reliance on NCAA tourney success, though levels of competition vary by conference
8. # of Conference regular season titles: see above
9. Total # of wins: another aggregate measure
10. # of NBA draft picks: in my opinion this may be a good measure from a recruit’s perspective, but from a fan’s perspective it doesn’t matter as much
11. # of NBA first round picks: see above

Basically, these metrics fall into 4 categories: NCAA Tournament performance, Conference performance, wins, and draft picks. I personally think that the NCAA tourney is king in this sport so the first 5 or 6 metrics should be weighted the highest. I suspect that most Uconn fans would agree…

The model works like this: choose a weighting for each of the 11 metrics (adding up to 100%), then choose a timeframe in years (anything from 1-30 years since my data goes back to 1985). The tool does the rest. It ranks each team according to those metrics and creates a composite score which is the weighted average of those ranks. Lowest composite score wins.

So what’s the final verdict? I took two cracks at this, one measuring NCAA tournament performance and another measuring all-around performance (both are attached). The NCAA tournament version places a 50% weighting on titles, 25% on final 4s, 15% on sweet 16s, and 10% on aggregate tourney wins. With these weightings Uconn ranks as the #2 program over the past 20 years, behind only Kentucky and ahead of (in order) UNC, Dook, Kansas, Mich St, FL, UCLA, AZ, Syracuse, Louisville, Indiana. The timeframe matters, but not that much. Some timeframes will make Uconn #1, some will make them 3 or 4. But no timeframe between 10-30 years puts Uconn any worse than #5, and within 25 years we are 4 or better.

The all-around version is weighted 35% titles, 20% final 4s, 10% sweet 16s, 10% conference tourney titles, 15% conference regular season titles, 10% total wins. In this version Uconn ranks #3 over the past 20 years behind Kentucky and Dook but ahead of UNC and Kansas who round out the top 5. Once again this result is not too time-sensitive, as Uconn ranks somewhere between #2 and #5 for all timeframes between 11-30 years.

So no matter how you cut it, Uconn is somewhere in the top 5. Often they are in the top 2 or 3.

I have attached the output for each version. In the lower left of each page are the weightings by metric. The top table shows the ranking of each team over the past 20 years for each and every metric – so you can use whatever weightings you want to create your own composite. The bottom chart shows Uconn’s composite rank for each timeframe between 1-30 years.

Finally, this data is current as of today. Therefore this year's Final 4 is reflected, but of course there is no champion yet for 2014. I will update this after next Monday - maybe by then UConn will be #1 on all versions!!!

Sorry for such a long post. Any thoughts?

 

[Tk79]

here is the all around version

 

[Gringo]

This is pretty interesting. Just out of curiosity what does the data say if you give an equal weight of 9% to each category?

 

[Tk79]

equal weight for all categories puts UConn at 5th place over 20 years behind Kentucky, Kansas, Duke, UNC. 6th place if you go back 30 years.

The two metrics that hurt us the most are NCAA tournament appearances (9th place) and, believe it or not, # of NBA draft picks (7th place). NCAA appearances because we've had several years missing the tourney, whereas most of these peers make it almost every year

 

[ConnHuskBask]

Cool data. I'm not too shocked regarding the tourney appearances hurting UConn but the draft picks were surprising.

I guess we don't have as many guys drafted but they certainly seem to stick around in the league.

Really the only two guys that washed out way quicker than anticipated were Marcus Williams and Josh Boone.

 

[Dogdeacon]

Unfortunately, I think you have to include what conference a school is in somewhere in your rankings. There is a reason they call them power conferences and why UConn is in a precarious state at the moment. Being on the fringe makes UConn's work harder on recruiting, scheduling and ultimately maintaining its status as a basketball power. The good news is the past and present achievements speak VOLUMES about the UConn athletic programs value for a conference and UConn's ability to overcome & succeed in the current environment.

 

[Gringo]

equal weight for all categories puts UConn at 5th place over 20 years behind Kentucky, Kansas, Duke, UNC. 6th place if you go back 30 years.

The two metrics that hurt us the most are NCAA tournament appearances (9th place) and, believe it or not, # of NBA draft picks (7th place). NCAA appearances because we've had several years missing the tourney, whereas most of these peers make it almost every year

Thanks for this. I like the analysis. I don't think anyone would look at all categories equally, since championships and final 4's are more important. I was just curious what the data showed. No matter how you slice the data we have one of the best programs in the country and in my opinion the best over the last 15 years.

 

[Tk79]

I agree with your point. All conference titles are not made equal. Likewise, the "wins" metric is impacted by conference and strength of schedule so not necessarily apples to apples. That said, I hesitate to start making adjustments to account for conference affiliations because it's not clear how one would do that without creating controversy. This is supposed to be an objective analysis, though admittedly based on subjective assumptions.

Frankly, the NCAA tournament is the great equalizer in my opinion and is the most objective category of performance criteria. It's also the most important. That is why I think the rankings based on tournament performance are the best indicators.

But your point is valid - being in a perceived weak conference may make things tougher going forward. But clearly not this year, and if we continue to perform in March and April, who cares?

 

[Onions]

So how does our program stack up relative to the other elite ones? I posted an analysis on this subject several years ago. I was going to post an update back in January as a dose of perspective when many of us (myself included) were down in the dumps on Uconn’s performance, but I was too lazy. Then we went on a run in March and we all got excited again. My inner voice keeps telling me to wait until after next weekend to post this, but I can’t get any work done and I’m tired of reading all the Boneyard threads for the third time, so I decided to make my own.

The question is simple – how do the top programs compare in terms of performance? The answer is not so simple because it is inherently a subjective topic and the criteria/definition of “performance” are not universal. For example, how much weight should national championships carry? What about final 4s, conference titles, wins? The timeframe is also a subjective input into any analysis like this. Is it a 10 year analysis? 20 years? 25? Or does even longer term historical success count just as much (the UCLA argument)?

There is no right answer here, which is why it’s a fun debate. But whatever argument you use, it must be based on facts. I did some research to collect the relevant facts, put them into a tool that allows you to choose your own criteria of performance, then math takes over and spits out the rankings of each program.

So here’s how it works. I chose a peer set of 12 teams to compare. Besides Uconn, I included the following programs: UNC, Dook, Kentucky, UCLA, Kansas, Indiana, Mich St, Arizona, Florida, Louisville. I also included Syracuse for sh@*s and giggles, but unfortunately for them regular season t-shirts didn’t qualify as one of my performance criteria.

As for the metrics, I chose (arbitrarily) 11 of them. I don’t think they are all equally important (more on this later) but I wanted a flexible tool to accommodate differing opinions. Metrics included are

1. 
   [ ]# of National titles: the ultimate measure of success
   [ ]# of final 4s: another commonly used indicator of “elite” seasons
   [ ]# of elite 8s
   [ ]# of Sweet 16s: rewards teams that are consistently solid, if not spectacular, in the tournament
   [ ]# of NCAA appearances: I don’t like this one for elite programs, more useful for lesser programs
   [ ]# of Tourney wins: an aggregate rather than seasonal measure
   [ ]# of Conference tourney titles: need something to balance out reliance on NCAA tourney success, though levels of competition vary by conference
   [ ]# of Conference regular season titles: see above
   [ ]Total # of wins: another aggregate measure
   [ ]# of NBA draft picks: in my opinion this may be a good measure from a recruit’s perspective, but from a fan’s perspective it doesn’t matter as much
   [ ]# of NBA first round picks: see above

Basically, these metrics fall into 4 categories: NCAA Tournament performance, Conference performance, wins, and draft picks. I personally think that the NCAA tourney is king in this sport so the first 5 or 6 metrics should be weighted the highest. I suspect that most Uconn fans would agree…

The model works like this: choose a weighting for each of the 11 metrics (adding up to 100%), then choose a timeframe in years (anything from 1-30 years since my data goes back to 1985). The tool does the rest. It ranks each team according to those metrics and creates a composite score which is the weighted average of those ranks. Lowest composite score wins.

So what’s the final verdict? I took two cracks at this, one measuring NCAA tournament performance and another measuring all-around performance (both are attached). The NCAA tournament version places a 50% weighting on titles, 25% on final 4s, 15% on sweet 16s, and 10% on aggregate tourney wins. With these weightings Uconn ranks as the #2 program over the past 20 years, behind only Kentucky and ahead of (in order) UNC, Dook, Kansas, Mich St, FL, UCLA, AZ, Syracuse, Louisville, Indiana. The timeframe matters, but not that much. Some timeframes will make Uconn #1, some will make them 3 or 4. But no timeframe between 10-30 years puts Uconn any worse than #5, and within 25 years we are 4 or better.

The all-around version is weighted 35% titles, 20% final 4s, 10% sweet 16s, 10% conference tourney titles, 15% conference regular season titles, 10% total wins. In this version Uconn ranks #3 over the past 20 years behind Kentucky and Dook but ahead of UNC and Kansas who round out the top 5. Once again this result is not too time-sensitive, as Uconn ranks somewhere between #2 and #5 for all timeframes between 11-30 years.

So no matter how you cut it, Uconn is somewhere in the top 5. Often they are in the top 2 or 3.

I have attached the output for each version. In the lower left of each page are the weightings by metric. The top table shows the ranking of each team over the past 20 years for each and every metric – so you can use whatever weightings you want to create your own composite. The bottom chart shows Uconn’s composite rank for each timeframe between 1-30 years.

Finally, this data is current as of today. Therefore this year's Final 4 is reflected, but of course there is no champion yet for 2014. I will update this after next Monday - maybe by then UConn will be #1 on all versions!!!

Sorry for such a long post. Any thoughts?

The NBA metric should include number of years those players played in the league, which is a better indication of actual quality of player and less about their athleticism. KO should count for 13 years as opposed to the zero he know accounts for, Jeff Adrien should be accounted for, as he is probably going to have a long run. Draft slot is about projected ceilings for a player, usually emphasizing athleticism. Years in the league represents heart and skill, which is where Uconn shines.

 

[tzznandrew]

I agree with your point. All conference titles are not made equal. Likewise, the "wins" metric is impacted by conference and strength of schedule so not necessarily apples to apples. That said, I hesitate to start making adjustments to account for conference affiliations because it's not clear how one would do that without creating controversy. This is supposed to be an objective analysis, though admittedly based on subjective assumptions.

Frankly, the NCAA tournament is the great equalizer in my opinion and is the most objective category of performance criteria. It's also the most important. That is why I think the rankings based on tournament performance are the best indicators.

But your point is valid - being in a perceived weak conference may make things tougher going forward. But clearly not this year, and if we continue to perform in March and April, who cares?

Agreed. And not all power conferences are created equal. There were down BE years where it was easy to rack up wins, and there were great years where it was difficult. The Pac 12 was in a down cycle for a while, and seem like they're coming out of it a bit. The SEC has been in a down cycle for longer than I can remember (or, really, care to look up).

The American had 5 good teams and 5 awful teams. If you were 1 of the top 5 teams, you had 8 tough games and 10 "easy" games. The SEC had 1 great team (Florida), 1 talented but underperforming team (Kentucky), 1 team that wasn't good but figured it out near the end (Tennessee), 2 decent teams (Mizzou and Arkansas), and 9 really bad teams. If you were one of the 3 NCAA teams, you played, at best 4 good games, 4 decent games, and 8 "easy" games.

 

[Tk79]

The NBA metric should include number of years those players played in the league, which is a better indication of actual quality of player and less about their athleticism. KO should count for 13 years as opposed to the zero he know accounts for, Jeff Adrien should be accounted for, as he is probably going to have a long run. Draft slot is about projected ceilings for a player, usually emphasizing athleticism. Years in the league represents heart and skill, which is where Uconn shines.

I will happily add this, if someone can just show me where to get the data

 

[tykurez]

I will happily add this, if someone can just show me where to get the data

Or perhaps remove it altogether. I have to imagine it skews programs like Kentucky who, in reality, don't really "develop" players so much anymore. It feels like that's truly too subjective.

Love the work you put into this though!

 

[Tk79]

Or perhaps remove it altogether. I have to imagine it skews programs like Kentucky who, in reality, don't really "develop" players so much anymore. It feels like that's truly too subjective.

Love the work you put into this though!

I agree - draft picks actually have no impact on my analysis because I put a weighting of 0% on those metrics. When I did this several years ago someone mentioned draft picks as a criteria, which is why I built in the flexibility to include it.

 

[Tk79]

if I were a recruit, then perhaps the draft metrics (and/or NBA longevity as some have suggested) would be very relevant. But as a fan, I think it's the team performance that really matters. Wins, conference titles, and by far most important, NCAA tourney performance

 

[Hans Sprungfeld]

I agree - draft picks actually have no impact on my analysis because I put a weighting of 0% on those metrics. When I did this several years ago someone mentioned draft picks as a criteria, which is why I built in the flexibility to include it.

Thanks. In the one scenario that dropped UCONN to 6th,whi entered the top 5. Also, if Draft #s are a dubious but included measurement, I also wonder how McD's AAs might factor in. The pattern I'm looking for includes any 'worst case view' in which I'm pretty sure UCONN does quite fine anyway.

 

[Tk79]

Thanks. In the one scenario that dropped UCONN to 6th,whi entered the top 5. Also, if Draft #s are a dubious but included measurement, I also wonder how McD's AAs might factor in. The pattern I'm looking for includes any 'worst case view' in which I'm pretty sure UCONN does quite fine anyway.

To answer your question - the one scenario that put us 6th - keep in mind that's if you equal weight every single metric, and even then only if you go back 26 years or longer. Arizona is the one who sneaks in at #5 here, probably because of their final 4 in '88.

I like your question about 'worst case' view - and in my opinion our worst case ranking is #5 because we generally fall anywhere from 1-6 depending on assumptions, and most often in the 2-5 range. There are virtually infinite possibilities for cutting the data, and the last thing I wanted to do was cherry pick a methodology and timeframe to make UConn look good. If that was my goal, I assure you UConn would have been #1. In fact, my "tourney performance" weightings would put UConn #1 under several different timeframes. But that's not a robust result so I wouldn't really consider Uconn the top performing program over the past 15, 20, 25 years. That distinction goes to either Duke or Kentucky. But we are for sure in the top 5, and you could even argue in the top 3 or 4.

 

[Kitaman]

For the ADHD or patience challenged, can the OP's post be summarized?

 

[Tk79]

For the ADHD or patience challenged, can the OP's post be summarized?

I'll take a crack at it because I can't stay away from this thread. Brevity is not my strong point though..

Summary: no matter how you analyze it, the top 5 programs over the past 10-25 years are Kentucky, Duke, UConn, UNC, and Kansas. The order changes depending on specific assumptions and timeframes, but those are the undisputed top 5.

 

[itgoeslike]

Good stuff. But, I'm curious as to why you didn't provide any weight for an elite eight performance while providing weight for a sweet sixteen. An elite 8 tempers the benefit of high seeding which typically helps teams through the first two rounds. I would have weighted it 40/20/10/5 for the NC/FF/EE/SS. Just my two cents. . .

 

[kobe]

I don't mean to hijack your thread but I actually did a similar analysis of the top programs since 2000. Now, this was done before this season so the numbers will be different.

Here was my methodology:
Conference Regular Season Title (3 points*1/(AVG RPI over period)- If tie all get the 3 points
Conference Tournament Title (2 points*1/AVG RPI over period)
NCAA Tournament Bid (3 points)
Ends in Sweet 16 (5 points)
Ends in Elite 8 (8 points)
Ends in Final Four (13 points)
Ends in Title Game (20 points)
National Champion (27 points)

I added a qualifier for the conference regular season and tournament so winning a weak A-10 wasn't equivalent to winning the old Big East. I compiled average RPI over the years considered and ranked the conference 1-9. Realignment was also considered. For example, Cincinnati winning C-USA is 2001 was not equivalent to Memphis winning C-USA in 2012.

These were the top 30:

 

[Tenspro2002]

Another possible metric that hasn't been discussed is NCAA Tournament seeding. Obviously, that's not a measure of NCAA Tournament results, but it does give an impression of overall regular season performance, and neutralizes some of the strength-of-schedule issues that something like regular season wins doesn't appropriately account for.

 

[tzznandrew]

I don't mean to hijack your thread but I actually did a similar analysis of the top programs since 2000. Now, this was done before this season so the numbers will be different.

Here was my methodology:
Conference Regular Season Title (3 points*1/(AVG RPI over period)- If tie all get the 3 points
Conference Tournament Title (2 points*1/AVG RPI over period)
NCAA Tournament Bid (3 points)
Ends in Sweet 16 (5 points)
Ends in Elite 8 (8 points)
Ends in Final Four (13 points)
Ends in Title Game (20 points)
National Champion (27 points)

I added a qualifier for the conference regular season and tournament so winning a weak A-10 wasn't equivalent to winning the old Big East. I compiled average RPI over the years considered and ranked the conference 1-9. Realignment was also considered. For example, Cincinnati winning C-USA is 2001 was not equivalent to Memphis winning C-USA in 2012.

These were the top 30:

I get that you picked 2000, as it makes sense to do the "new millenium," but doing that 14 year window hurts UConn, since taking away that one year negates a regular season title, tournament title, and NCAA title. I'd wager that'd bump UConn up a bit.

I also think there's probably too big a gap between losing in the Final Four and losing in the championship game. No one cares who is the runner-up. They care that you won, or that you made the Final Four. Sure, being the runner up brings some caché, but not as much as this model suggests.

But, to be fair, all these dates are arbitrary in some way. And this is pretty cool.

 

[Tk79]

Interesting analysis. Some interesting stuff in there, especially regarding adjustments for RPI and whatnot.

I think our two approaches, while similar, are slightly different in intent. I was not trying to create a perfect rankings formula - on the contrary I would argue that one doesn't exist. My main objective was to see how robust UConn's rating is to different methodologies - i.e. if you change the timeframe and the weightings how much does it move things. My conclusion was, happily, that UConn is in the top 5 (almost) no matter what you do.

To reiterate a comment on your methodology - starting in 2000 is a disadvantage to UConn because you could start one year earlier and we'd have another title in the mix. Also, my approach includes this years FF which of course benefits us...

 

[Tk79]

Good stuff. But, I'm curious as to why you didn't provide any weight for an elite eight performance while providing weight for a sweet sixteen. An elite 8 tempers the benefit of high seeding which typically helps teams through the first two rounds. I would have weighted it 40/20/10/5 for the NC/FF/EE/SS. Just my two cents. . .

fair point. My weightings are subjective. This thread has already passed the information overload threshold so I won't add a whole new analysis - but suffice to say your suggestion of including Elite 8s would improve the results for UConn. We have the 3rd most elite 8s over the past 20-25 years behind Kentucky and UNC