Big East NET Rankings

[HooperScooper]

You’re right

Do you want UConn to go back to the American?

 

[ucp990411]

LOL BCU and Rutgers

 

[auror]

What do you mean by this description?

About the NET's formula, the NCAA says "Factors include the Team Value Index (TVI), which is a result-based feature that rewards teams for beating quality opponents, particularly away from home, as well as an adjusted net efficiency rating. The adjusted efficiency is a team’s net efficiency, adjusted for strength of opponent and location (home/away/neutral) across all games played. "

We don't know the exact formula and weighting for the NET, but there are people who have reverse-engineered it quite closely. In their calculations, the efficiency component is worth roughly 80% of the overall team score, and the Team Value Index 20%. So 4:1 efficiency:resume components. Whereas something like KenPom is 100% efficiency and SOR or WAB is 100% resume (the only efficiencies that matter are your opponents'). You can consider KenPom to be predictive, the resume models to be backwards-looking, and the NET a hybrid which leans more towards predictive.

 

[nelsonmuntz]

About the NET's formula, the NCAA says "Factors include the Team Value Index (TVI), which is a result-based feature that rewards teams for beating quality opponents, particularly away from home, as well as an adjusted net efficiency rating. The adjusted efficiency is a team’s net efficiency, adjusted for strength of opponent and location (home/away/neutral) across all games played. "

We don't know the exact formula and weighting for the NET, but there are people who have reverse-engineered it quite closely. In their calculations, the efficiency component is worth roughly 80% of the overall team score, and the Team Value Index 20%. So 4:1 efficiency:resume components. Whereas something like KenPom is 100% efficiency and SOR or WAB is 100% resume (the only efficiencies that matter are your opponents'). You can consider KenPom to be predictive, the resume models to be backwards-looking, and the NET a hybrid which leans more towards predictive.

Is the "overall team score" adjusted for quality of opponent, or is the 20% TVI the only way that opponent quality is factored in at all?

 

[auror]

Is the "overall team score" adjusted for quality of opponent, or is the 20% TVI the only way that opponent quality is factored in at all?

The net efficiency is heavily adjusted for opponent, location, and pace as well.

The neutral site games that are pseudo-home games are the best way to game the NET right now.

 

[RuffRuff]

Do you want UConn to go back to the American?

I'd rather sit on a unicycle missing the seat. The Ollie/AAC days were as bleak as it could possibly get. I appreciate the BE for that, but it does stink seeing them sink in status/competitiveness. I personally don't feel like it will change much, but I'd be happy to be wrong.

 

[nelsonmuntz]

The net efficiency is heavily adjusted for opponent, location, and pace as well.

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

 

[RuffRuff]

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

Is anyone using the RPI anymore? I feel like the newer analytics are more accurate (KP, Torvik, Maya) .

 

[nelsonmuntz]

Is anyone using the RPI anymore? I feel like the newer analytics are more accurate (KP, Torvik, Maya) .

I am pointing out a problem with the newer analytics, which are not nearly as clever as they make themselves out to be.

 

[auror]

The RPI had problems, but the NET treats a 50 point blowout of the 350th ranked team in D1 as somehow relevant and additive to a Top 25 team's ranking, which is a big flaw in the system. The RPI would punish tournament teams for playing too many of those games, but now, as long as the favorite wins by enough, these games help a team's tournament resume.

It's true, but relatively speaking it's hard to beat even the 350th team by 50. If a lot of other teams beat them by more than you did per possession (say most teams beat them by 55 and you beat them by 50), your rating will still go down.

 

[nelsonmuntz]

It's true, but relatively speaking it's hard to beat even the 350th team by 50. If a lot of other teams beat them by more than you did per possession (say most teams beat them by 55 and you beat them by 50), your rating will still go down.

Why should it matter once a team is winning by more than, say, 25? A blowout is a blowout, the game was not competitive, and why should winning by 55 instead of 35 have any impact on a team's rating? What meaningful information does the difference tell us about that team? The only thing the favorite in a game like that should be thinking about is giving its bench minutes and avoiding injuries.

 

[auror]

Why should it matter once a team is winning by more than, say, 25? A blowout is a blowout, the game was not competitive, and why should winning by 55 instead of 35 have any impact on a team's rating? What meaningful information does the difference tell us about that team? The only thing the favorite in a game like that should be thinking about is giving its bench minutes and avoiding injuries.

Because they've analyzed tens of thousands of games and have found that using the real margins above 25 are still more predictive instead of capping the margin in the model.

Edit- There are likely diminishing returns and most systems have some mitigating factors for gross mismatches, but in general there is signal in the data.

 

[RuffRuff]

 

[Blades]

Teams by conference in top 50 (click image)

 

[FfldCntyFan]

Teams by conference in top 50 (click image)

View attachment 115402

Why do you want to go back to the AAC?

 

[businesslawyer]

I am pointing out a problem with the newer analytics, which are not nearly as clever as they make themselves out to be.

So here’s the thing. It turns out that, as a matter of making predictions, it is useful to know and account for margin of victory because in practice a team that wins every game by 40 is likely to beat a team that, against the same schedule, wins every game by 20. And, if you’re a better, you’re looking to benefit from that type of predictive metric. (How clever a predictive formula is depends on what the formula is, but every one we use in some ways adjusts for quality of opponent and location of game.)

A resume based system only asks who you beat and who you lose to and how good they are and where are the games. If you go back to my example in the prior paragraph, a resume system would have those two teams equal, even though the predictive system would have the team that wins by an extra 20 points a game substantially ahead. If I were on the committee I would be much more concerned with resume metrics if , as I don’t care about how much you win or lose by because the only thing that matters is whether you win or lose. No professional sports cares about margin of victory, except to break ties.


By the way, Nelson, I know you know all this. The truth is that we really don’t know exactly how clever or not clever any computer formula ranking is. But we can judge whether we should be using predictive metrics, resume metrics or a blend.

And no. By trying to explain the different types of computer programs I am NOT saying I’d rather be playing in the American.

 

[Blades]

Why do you want to go back to the AAC?

Nah...You must mean someone else... I was telling Uconn fans over and over on TOS they needed to get out of the American," years before
the school finally did" and returned to the BE....next up ....#ACC/ # Big 12 or bust ..... 😉

 

[nelsonmuntz]

So here’s the thing. It turns out that, as a matter of making predictions, it is useful to know and account for margin of victory because in practice a team that wins every game by 40 is likely to beat a team that, against the same schedule, wins every game by 20. And, if you’re a better, you’re looking to benefit from that type of predictive metric. (How clever a predictive formula is depends on what the formula is, but every one we use in some ways adjusts for quality of opponent and location of game.)

A resume based system only asks who you beat and who you lose to and how good they are and where are the games. If you go back to my example in the prior paragraph, a resume system would have those two teams equal, even though the predictive system would have the team that wins by an extra 20 points a game substantially ahead. If I were on the committee I would be much more concerned with resume metrics if , as I don’t care about how much you win or lose by because the only thing that matters is whether you win or lose. No professional sports cares about margin of victory, except to break ties.


By the way, Nelson, I know you know all this. The truth is that we really don’t know exactly how clever or not clever any computer formula ranking is. But we can judge whether we should be using predictive metrics, resume metrics or a blend.

And no. By trying to explain the different types of computer programs I am NOT saying I’d rather be playing in the American.

I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results. Individual coaching choices unrelated to the outcome of the game will play a big factor in the ultimate margin of victory, since by definition these games are not competitive.

One of the problems with all the "predictive" models is that they are a black box, which I don't care about if they are just used for fun or by gamblers, since those are two instances of "buyer beware" and not a problem for the vast majority of us. But when they are used for evaluative purposes for seeding and access to the NCAA Tournament, then they should be transparent with the evaluation criteria. Instead, the NCAA says "trust us", which is problematic given the dollars at stake and the NCAA's track record. Evaluation criteria should be available to those being evaluated or impacted by the criteria.

Without knowing what makes up these models, is there at least some kind of independent service evaluating the models? How do we know that the models are not garbage? Have they been back-tested and is that information available, or is there at least a report showing that someone else back-tested the models to show that they worked?

 

[willie99]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

Metrics

 

[auror]

I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results. Individual coaching choices unrelated to the outcome of the game will play a big factor in the ultimate margin of victory, since by definition these games are not competitive.
...
Without knowing what makes up these models, is there at least some kind of independent service evaluating the models? How do we know that the models are not garbage? Have they been back-tested and is that information available, or is there at least a report showing that someone else back-tested the models to show that they worked?

You haven't done the research, so to be blunt, your "disagreement" is pointless, meritless, and frankly the hubris is comical. Every possession tells you something, even games that are less competitive. Some possessions tell you more. The good models understand this distinction and employ various techniques to refine. "Significant drivers of the models' results" is your own interpretation of what is happening.

There are certainly people and sites that evaluate models. Try a google. You can start here if you want.

Ranking Analysis

 

[bendm]

The neutral site games that are pseudo-home games are the best way to game the NET right now.

ie the OOCs in NYC and Boston. Now if only we could stop with the 300+ buy games and we’d have it down pat.

 

[HooperScooper]

ie the OOCs in NYC and Boston. Now if only we could stop with the 300+ buy games and we’d have it down pat.

Hurley said recently he'd like to only play 3 buy games next season. Not sure that's going to happen but we'll see.

"It's gone so well, next year we may go to eight of these big games, and three buys, because I hate the buy games," Hurley told CBS Sports after UConn's 77-73 win over Florida earlier this week. "When I wake up the day of a buy game, I just want to go die. The anxiety, the fear that your team is — the wrecking of a loss, or just how mad you get at your team when they underestimate a scrappy, loaded mid-major team. I hate those games."

Dan Hurley wants to make UConn's nonconference schedule even tougher next season

Hurley told CBS Sports he also hopes the Big East goes from a 20- to an 18-game schedule to free up more scheduling flexibility

www.cbssports.com

 

[jleves]

It’s amazing to me that based on our resume which seems to me to be a top 3 resume of all teams in college ball at worst that the net has us 8th. We are very clearly one of the best 3-4 teams this year

Part of that 8th ranking is we haven't had our full complement of players for most of our games so we were not winning by bigger margins. Healthier team, bigger margins, higher ranking. Also, teams with no losses will gain some NET points.

We're fine at 8 right now. It's not the only thing that is used and we fortunately have the opportunity to climb up.

 

[nelsonmuntz]

You haven't done the research, so to be blunt, your "disagreement" is pointless, meritless, and frankly the hubris is comical. Every possession tells you something, even games that are less competitive. Some possessions tell you more. The good models understand this distinction and employ various techniques to refine. "Significant drivers of the models' results" is your own interpretation of what is happening.

There are certainly people and sites that evaluate models. Try a google. You can start here if you want.

Ranking Analysis

I used punctuation symbols "?" that are called question marks and meant to indicate that the sentence is a question, and a request for information. There is no need to get upset when I am simply asking if anyone has actually reviewed these models.

The analysis in your link shows:

1) That the models appear to be tightly correlated to each other, which cuts both ways in assessing their credibility and usefulness.
2) There are no benchmarks included making that analysis meaningless. Did the models do better or worse than a more straightforward analysis like RPI?
3) It is likely that a significant percentage of games have easily predictable outcomes, which creates a confirmation bias in an analysis like that one.
4) NET was not assessed in that analysis.

I am not a statistician, but I have a little game when it comes to assessing models since I have spent most of my career in and around technology and financial services. The link you provided is helpful if these models were just being used for entertainment purposes, but is not remotely acceptable as a validation tool for the accuracy of any model or algorithm that is used for major decisions that have significant financial impacts. There are dozens (or possibly hundreds when you include independent contractors) of credible firms that could do an assessment of the accuracy and predictive value of the NET algorithm, because they do stuff like this all the time for all kinds of companies. Did the NCAA engage one or more of these firms?

I have not seen any reference to an independent validation of NET. This shouldn't be controversial. This is basic stuff that any large entity should do for a key algorithm whose outcome has a significant financial impact on third parties. Was it done?

 

[auror]

I used punctuation symbols "?" that are called question marks and meant to indicate that the sentence is a question, and a request for information. There is no need to get upset when I am simply asking if anyone has actually reviewed these models.

The analysis in your link shows:

1) That the models appear to be tightly correlated to each other, which cuts both ways in assessing their credibility and usefulness.
2) There are no benchmarks included making that analysis meaningless. Did the models do better or worse than a more straightforward analysis like RPI?
3) It is likely that a significant percentage of games have easily predictable outcomes, which creates a confirmation bias in an analysis like that one.
4) NET was not assessed in that analysis.

I am not a statistician, but I have a little game when it comes to assessing models since I have spent most of my career in and around technology and financial services. The link you provided is helpful if these models were just being used for entertainment purposes, but is not remotely acceptable as a validation tool for the accuracy of any model or algorithm that is used for major decisions that have significant financial impacts. There are dozens (or possibly hundreds when you include independent contractors) of credible firms that could do an assessment of the accuracy and predictive value of the NET algorithm, because they do stuff like this all the time for all kinds of companies. Did the NCAA engage one or more of these firms?

I have not seen any reference to an independent validation of NET. This shouldn't be controversial. This is basic stuff that any large entity should do for a key algorithm whose outcome has a significant financial impact on third parties. Was it done?

There was no question mark in your statement: "I simply disagree with the principle that different levels of blowouts are meaningfully predictive or that they should be significant drivers of the models' results." That was entirely where my first paragraph was directed, which I used punctuation symbols known as "quotation marks" to reference your "disagree" statement so that you would have the context clues to understand that. I'm not upset, I said it was comical. You "disagreeing" with a "principle" with absolutely nothing to back it up but your good old gut feeling and contrarian can-do attitude makes me chuckle.

I proceeded to then help you start getting answers to some of your questions. But the key phrases I used were "start here" and "do a google". My link definitely does not answer everything. The answers to the thoughts you posed with your "question mark punctuation symbols" and the subsequent ones in your follow-up are probably all out there if you're interested enough. Good luck. Start your research. The only thing I'll answer is that of course all modern models that are empirically-derived and back-tested do better than the RPI. The RPI is unsound and arbitrary. That and yes the NET is on that site, but you have to uncheck the hide box since it doesn't release at the start of the season.