Big East NET Rankings

[RuffRuff]

Do you want UConn to go back to the American?

I'd rather sit on a unicycle missing the seat. The Ollie/AAC days were as bleak as it could possibly get. I appreciate the BE for that, but it does stink seeing them sink in status/competitiveness. I personally don't feel like it will change much, but I'd be happy to be wrong.

 

[nelsonmuntz]

The net efficiency is heavily adjusted for opponent, location, and pace as well.

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

 

[RuffRuff]

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

Is anyone using the RPI anymore? I feel like the newer analytics are more accurate (KP, Torvik, Maya) .

 

[nelsonmuntz]

Is anyone using the RPI anymore? I feel like the newer analytics are more accurate (KP, Torvik, Maya) .

I am pointing out a problem with the newer analytics, which are not nearly as clever as they make themselves out to be.

 

[auror]

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

It's true, but relatively speaking it's hard to beat even the 350th team by 50. If a lot of other teams beat them by more than you did per possession (say most teams beat them by 55 and you beat them by 50), your rating will still go down.

 

[nelsonmuntz]

It's true, but relatively speaking it's hard to beat even the 350th team by 50. If a lot of other teams beat them by more than you did per possession (say most teams beat them by 55 and you beat them by 50), your rating will still go down.

Why should it matter once a team is winning by more than, say, 25? A blowout is a blowout, the game was not competitive, and why should winning by 55 instead of 35 have any impact on a team's rating? What meaningful information does the difference tell us about that team? The only thing the favorite in a game like that should be thinking about is giving its bench minutes and avoiding injuries.

 

[auror]

Why should it matter once a team is winning by more than, say, 25? A blowout is a blowout, the game was not competitive, and why should winning by 55 instead of 35 have any impact on a team's rating? What meaningful information does the difference tell us about that team? The only thing the favorite in a game like that should be thinking about is giving its bench minutes and avoiding injuries.

Because they've analyzed tens of thousands of games and have found that using the real margins above 25 are still more predictive instead of capping the margin in the model.

Edit- There are likely diminishing returns and most systems have some mitigating factors for gross mismatches, but in general there is signal in the data.

 

[RuffRuff]

 

[Blades]

Teams by conference in top 50 (click image)

 

[FfldCntyFan]

Teams by conference in top 50 (click image)

View attachment 115402

Why do you want to go back to the AAC?

 

[businesslawyer]

I am pointing out a problem with the newer analytics, which are not nearly as clever as they make themselves out to be.

So here’s the thing. It turns out that, as a matter of making predictions, it is useful to know and account for margin of victory because in practice a team that wins every game by 40 is likely to beat a team that, against the same schedule, wins every game by 20. And, if you’re a better, you’re looking to benefit from that type of predictive metric. (How clever a predictive formula is depends on what the formula is, but every one we use in some ways adjusts for quality of opponent and location of game.)

A resume based system only asks who you beat and who you lose to and how good they are and where are the games. If you go back to my example in the prior paragraph, a resume system would have those two teams equal, even though the predictive system would have the team that wins by an extra 20 points a game substantially ahead. If I were on the committee I would be much more concerned with resume metrics if , as I don’t care about how much you win or lose by because the only thing that matters is whether you win or lose. No professional sports cares about margin of victory, except to break ties.


By the way, Nelson, I know you know all this. The truth is that we really don’t know exactly how clever or not clever any computer formula ranking is. But we can judge whether we should be using predictive metrics, resume metrics or a blend.

And no. By trying to explain the different types of computer programs I am NOT saying I’d rather be playing in the American.

 

[Blades]

Why do you want to go back to the AAC?

Nah...You must mean someone else... I was telling Uconn fans over and over on TOS they needed to get out of the American," years before
the school finally did" and returned to the BE....next up ....#ACC/ # Big 12 or bust ..... 😉

 

[nelsonmuntz]

So here’s the thing. It turns out that, as a matter of making predictions, it is useful to know and account for margin of victory because in practice a team that wins every game by 40 is likely to beat a team that, against the same schedule, wins every game by 20. And, if you’re a better, you’re looking to benefit from that type of predictive metric. (How clever a predictive formula is depends on what the formula is, but every one we use in some ways adjusts for quality of opponent and location of game.)

A resume based system only asks who you beat and who you lose to and how good they are and where are the games. If you go back to my example in the prior paragraph, a resume system would have those two teams equal, even though the predictive system would have the team that wins by an extra 20 points a game substantially ahead. If I were on the committee I would be much more concerned with resume metrics if , as I don’t care about how much you win or lose by because the only thing that matters is whether you win or lose. No professional sports cares about margin of victory, except to break ties.


By the way, Nelson, I know you know all this. The truth is that we really don’t know exactly how clever or not clever any computer formula ranking is. But we can judge whether we should be using predictive metrics, resume metrics or a blend.

And no. By trying to explain the different types of computer programs I am NOT saying I’d rather be playing in the American.

I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results. Individual coaching choices unrelated to the outcome of the game will play a big factor in the ultimate margin of victory, since by definition these games are not competitive.

One of the problems with all the "predictive" models is that they are a black box, which I don't care about if they are just used for fun or by gamblers, since those are two instances of "buyer beware" and not a problem for the vast majority of us. But when they are used for evaluative purposes for seeding and access to the NCAA Tournament, then they should be transparent with the evaluation criteria. Instead, the NCAA says "trust us", which is problematic given the dollars at stake and the NCAA's track record. Evaluation criteria should be available to those being evaluated or impacted by the criteria.

Without knowing what makes up these models, is there at least some kind of independent service evaluating the models? How do we know that the models are not garbage? Have they been back-tested and is that information available, or is there at least a report showing that someone else back-tested the models to show that they worked?

 

[willie99]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

Metrics

 

[auror]

I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results. Individual coaching choices unrelated to the outcome of the game will play a big factor in the ultimate margin of victory, since by definition these games are not competitive.
...
Without knowing what makes up these models, is there at least some kind of independent service evaluating the models? How do we know that the models are not garbage? Have they been back-tested and is that information available, or is there at least a report showing that someone else back-tested the models to show that they worked?

You haven't done the research, so to be blunt, your "disagreement" is pointless, meritless, and frankly the hubris is comical. Every possession tells you something, even games that are less competitive. Some possessions tell you more. The good models understand this distinction and employ various techniques to refine. "Significant drivers of the models' results" is your own interpretation of what is happening.

There are certainly people and sites that evaluate models. Try a google. You can start here if you want.

Ranking Analysis

 

[bendm]

The neutral site games that are pseudo-home games are the best way to game the NET right now.

ie the OOCs in NYC and Boston. Now if only we could stop with the 300+ buy games and we’d have it down pat.

 

[HooperScooper]

ie the OOCs in NYC and Boston. Now if only we could stop with the 300+ buy games and we’d have it down pat.

Hurley said recently he'd like to only play 3 buy games next season. Not sure that's going to happen but we'll see.

"It's gone so well, next year we may go to eight of these big games, and three buys, because I hate the buy games," Hurley told CBS Sports after UConn's 77-73 win over Florida earlier this week. "When I wake up the day of a buy game, I just want to go die. The anxiety, the fear that your team is — the wrecking of a loss, or just how mad you get at your team when they underestimate a scrappy, loaded mid-major team. I hate those games."

Dan Hurley wants to make UConn's nonconference schedule even tougher next season

Hurley told CBS Sports he also hopes the Big East goes from a 20- to an 18-game schedule to free up more scheduling flexibility

www.cbssports.com

 

[jleves]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

Part of that 8th ranking is we haven't had our full complement of players for most of our games so we were not winning by bigger margins. Healthier team, bigger margins, higher ranking. Also, teams with no losses will gain some NET points.

We're fine at 8 right now. It's not the only thing that is used and we fortunately have the opportunity to climb up.

 

[nelsonmuntz]

You haven't done the research, so to be blunt, your "disagreement" is pointless, meritless, and frankly the hubris is comical. Every possession tells you something, even games that are less competitive. Some possessions tell you more. The good models understand this distinction and employ various techniques to refine. "Significant drivers of the models' results" is your own interpretation of what is happening.

There are certainly people and sites that evaluate models. Try a google. You can start here if you want.

Ranking Analysis

I used punctuation symbols "?" that are called question marks and meant to indicate that the sentence is a question, and a request for information. There is no need to get upset when I am simply asking if anyone has actually reviewed these models.

The analysis in your link shows:

1) That the models appear to be tightly correlated to each other, which cuts both ways in assessing their credibility and usefulness.
2) There are no benchmarks included making that analysis meaningless. Did the models do better or worse than a more straightforward analysis like RPI?
3) It is likely that a significant percentage of games have easily predictable outcomes, which creates a confirmation bias in an analysis like that one.
4) NET was not assessed in that analysis.

I am not a statistician, but I have a little game when it comes to assessing models since I have spent most of my career in and around technology and financial services. The link you provided is helpful if these models were just being used for entertainment purposes, but is not remotely acceptable as a validation tool for the accuracy of any model or algorithm that is used for major decisions that have significant financial impacts. There are dozens (or possibly hundreds when you include independent contractors) of credible firms that could do an assessment of the accuracy and predictive value of the NET algorithm, because they do stuff like this all the time for all kinds of companies. Did the NCAA engage one or more of these firms?

I have not seen any reference to an independent validation of NET. This shouldn't be controversial. This is basic stuff that any large entity should do for a key algorithm whose outcome has a significant financial impact on third parties. Was it done?

 

[auror]

I used punctuation symbols "?" that are called question marks and meant to indicate that the sentence is a question, and a request for information. There is no need to get upset when I am simply asking if anyone has actually reviewed these models.

The analysis in your link shows:

1) That the models appear to be tightly correlated to each other, which cuts both ways in assessing their credibility and usefulness.
2) There are no benchmarks included making that analysis meaningless. Did the models do better or worse than a more straightforward analysis like RPI?
3) It is likely that a significant percentage of games have easily predictable outcomes, which creates a confirmation bias in an analysis like that one.
4) NET was not assessed in that analysis.

I am not a statistician, but I have a little game when it comes to assessing models since I have spent most of my career in and around technology and financial services. The link you provided is helpful if these models were just being used for entertainment purposes, but is not remotely acceptable as a validation tool for the accuracy of any model or algorithm that is used for major decisions that have significant financial impacts. There are dozens (or possibly hundreds when you include independent contractors) of credible firms that could do an assessment of the accuracy and predictive value of the NET algorithm, because they do stuff like this all the time for all kinds of companies. Did the NCAA engage one or more of these firms?

I have not seen any reference to an independent validation of NET. This shouldn't be controversial. This is basic stuff that any large entity should do for a key algorithm whose outcome has a significant financial impact on third parties. Was it done?

There was no question mark in your statement: "I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results." That was entirely where my first paragraph was directed, which I used punctuation symbols known as "quotation marks" to reference your "disagree" statement so that you would have the context clues to understand that. I'm not upset, I said it was comical. You "disagreeing" with a "principle" with absolutely nothing to back it up but your good old gut feeling and contrarian can-do attitude makes me chuckle.

I proceeded to then help you start getting answers to some of your questions. But the key phrases I used were "start here" and "do a google". My link definitely does not answer everything. The answers to the thoughts you posed with your "question mark punctuation symbols" and the subsequent ones in your follow-up are probably all out there if you're interested enough. Good luck. Start your research. The only thing I'll answer is that of course all modern models that are empirically-derived and back-tested do better than the RPI. The RPI is unsound and arbitrary. That and yes the NET is on that site, but you have to uncheck the hide box since it doesn't release at the start of the season.

 

[JwittersPAC]

I used punctuation symbols "?" that are called question marks and meant to indicate that the sentence is a question, and a request for information. There is no need to get upset when I am simply asking if anyone has actually reviewed these models.

The analysis in your link shows:

1) That the models appear to be tightly correlated to each other, which cuts both ways in assessing their credibility and usefulness.
2) There are no benchmarks included making that analysis meaningless. Did the models do better or worse than a more straightforward analysis like RPI?
3) It is likely that a significant percentage of games have easily predictable outcomes, which creates a confirmation bias in an analysis like that one.
4) NET was not assessed in that analysis.

I am not a statistician, but I have a little game when it comes to assessing models since I have spent most of my career in and around technology and financial services. The link you provided is helpful if these models were just being used for entertainment purposes, but is not remotely acceptable as a validation tool for the accuracy of any model or algorithm that is used for major decisions that have significant financial impacts. There are dozens (or possibly hundreds when you include independent contractors) of credible firms that could do an assessment of the accuracy and predictive value of the NET algorithm, because they do stuff like this all the time for all kinds of companies. Did the NCAA engage one or more of these firms?

I have not seen any reference to an independent validation of NET. This shouldn't be controversial. This is basic stuff that any large entity should do for a key algorithm whose outcome has a significant financial impact on third parties. Was it done?

I am not a statistician but I pretend to be an expert on judging statistical models…..lol

 

[Scrutineer]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

It puzzled me why Villanova has such a low NET so I looked at their schedule. Two things stood out. They only played a couple terrible teams while we played five and, while their schedule did not have as many powerhouse opponents, it did have a number of high level opponents.
So, at least based on the schedules UConn and Villanova have played, it appears we are penalized to a degree for playing so many Columbias and New Havens.

 

[auror]

It puzzled me why Villanova has such a low NET so I looked at their schedule. Two things stood out. They only played a couple terrible teams while we played five and, while their schedule did not have as many powerhouse opponents, it did have a number of high level opponents.
So, at least based on the schedules UConn and Villanova have played, it appears we are penalized to a degree for playing so many Columbias and New Havens.

Biggest games improving their net:
Road win at Seton Hall. Neutral win over Wisconsin by 10 (even though OT, I'm not positive how NET treats OT but many of the models just consider it as the real margin because possessions are possessions). Sacred Heart win by 34, and Penn win by 27.

The Michigan beatdown hurt them and the Old Dominion win wasn't great, but they haven't had any real stinkers.

 

[RuffRuff]

Biggest games improving their net:
Road win at Seton Hall. Neutral win over Wisconsin by 10 (even though OT, I'm not positive how NET treats OT but many of the models just consider it as the real margin because possessions are possessions). Sacred Heart win by 34, and Penn win by 27.

The Michigan beatdown hurt them and the Old Dominion win wasn't great, but they haven't had any real stinkers.

They played BYU fairly tough.

I think that needs to be a schedule adjustment - flipping to a higher degree of non high major. It’s probably better for the team and the fans.

 

[pepband99]

I am pointing out a problem with the newer analytics, which are not nearly as clever as they make themselves out to be.

No, you’re just pointing out that you still don’t understand how these metrics work.

To help you, try flipping your argument around. If, say, Duke, beat CCSU by 2 in Cameron, you wouldn’t find that result compelling? You don’t think it reflects on either teams resume?

Your precious RPI would treat that as any home win.

 

[RuffRuff]

No, you’re just pointing out that you still don’t understand how these metrics work.

To help you, try flipping your argument around. If, say, Duke, beat CCSU by 2 in Cameron, you wouldn’t find that result compelling? You don’t think it reflects on either teams resume?

Your precious RPI would treat that as any home win.

Clawing for supporting data points when logic and common sense prevail.

 

[NHUConnfan]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

And we haven't even really played very clean ball yet. I'm okay with that if it gives us tape to work on so we're humming by February.

 

[businesslawyer]

No, you’re just pointing out that you still don’t understand how these metrics work.

To help you, try flipping your argument around. If, say, Duke, beat CCSU by 2 in Cameron, you wouldn’t find that result compelling? You don’t think it reflects on either teams resume?

Your precious RPI would treat that as any home win.

So take that one step further. If the Yankees sweep a 3 game set against the Nationals at home, winning each game by 1 run, and the Red Sox sweep the Nationals at home winning each game by 10, would the margin be relevant in trying to predict who would win a Yankees--Red Sox series the following week? Of course it would be relevant. Not determinative, but certainly relevant.

On the other hand, no one would seriously propose that, in the standings, the Sox should get more wins for their sweep than the Yankees get from theirs. So when folks object to putting much weight on the KenPom/Net type numbers that's all they're saying. Because schedules in college sports are far less balanced than in a professional league, it's fine to reward and punish teams for their strengths of schedule when you compare their records, but just because margins are useful in making predictions doesn't mean they should be used in ranking teams for anything meaningful.

 

[RuffRuff]

So take that one step further. If the Yankees sweep a 3 game set against the Nationals at home, winning each game by 1 run, and the Red Sox sweep the Nationals at home winning each game by 10, would the margin be relevant in trying to predict who would win a Yankees--Red Sox series the following week? Of course it would be relevant. Not determinative, but certainly relevant.

On the other hand, no one would seriously propose that, in the standings, the Sox should get more wins for their sweep than the Yankees get from theirs. So when folks object to putting much weight on the KenPom/Net type numbers that's all they're saying. Because schedules in college sports are far less balanced than in a professional league, it's fine to reward and punish teams for their strengths of schedule when you compare their records, but just because margins are useful in making predictions doesn't mean they should be used in ranking teams for anything meaningful.

100% would be relevant. Scoring differential matters. A 30 point win vs a 50 point win is meaningful - it means a team is that much more dominant, focused and can play through a game without letting down. Kill shots.

 

[GlastonTim]

100% would be relevant. Scoring differential matters. A 30 point win vs a 50 point win is meaningful - it means a team is that much more dominant, focused and can play through a game without letting down. Kill shots.

Or it could mean a team is resting starters and giving bench an opportunity for minutes.