- This topic has 95 replies, 27 voices, and was last updated 9 years, 8 months ago by
.
-
Posts
-
2 teams play the exact same schedule. They have the exact same record. They have the exact same RPI.
Team 1 wins its game by an average margin of 10 versus an average margin of 5 for Team 2. Team 1 loses by an average margin of 5 versus 9.
The net average margin is 5 for Team 1 and 1 for Team 2.
Add in standard deviation of average margin to further adjust ranking. Consistency counts. Make your own RPI adjusted for net margin and volatility.
Sorry, but it sounds like you are arguing in circles or maybe I’m just not following you.
You claim that SMU’s schedule than the NCAAT gave them credit for. Your proof is a ranking in a different computer algorithm…with no specific explanation of why that algorithm is better. Which pretty much goes for your claims of using MoV instead of W/L records. I don’t know of any ranking system in any sport that uses anything other than W/L record.
(PS…determining who the good teams are is not the same thing as predicting who will win the games between those teams.)
Maybe there is a better algorithm than RPI. (But if SMU is your shining proof, then I’m not impressed.) I haven’t studied any of them because they aren’t used for anything real. I have spent my time trying to figure out how the Selection Committee reaches their decisions. If you want to tilt at windmills, then let me get out of your way,
I doubt you watched most of the games SMU played in their OOC schedule. I doubt you saw any games those teams played. You’re going by their W/L record, what you know about their conference,
^This is fair. I personally saw SMU twice on tv last season, and my impression was ‘meh’. I know that individual committee members make great efforts to see viable teams in person throughout the season. This personal accounting may have more sway than we know. It’s important to note that one of the games I saw SMU play was the Houston loss in the conference tournament … Not a good look, especially on selection weekend.
My overriding point was that regardless of objective criteria, selection will always be a subjective process. I thought the committee got this one correct. Given what was known of how the decisions would ultimately be made, I was not shocked/surprised at all when SMU was excluded:
– weak non-con/overall schedule
– non-con was bereft of quality results (did you challenge yourself?)
– good = swept UConn, wins over Memphis, Cincinnati
– bad losses = USF, Temple (Temple was 9-22 last season), Houston; one too many bad losses for this schedule
– lost three consecutive games to close the season^The bolded point is, imo, the deciding factor. Fair or not, closing the season in such a fashion gave the committee every excuse to use confirmation bias and exclude SMU. The Mustangs simply could not afford to lose that Houston game … they did.
The RPI rather tends to underrate teams from strong conferences and regions and to overrate teams from weak conferences and regions
^I recall the last time they tweaked the RPI, some of the ‘usual suspects’ (power conference coaches on the bubble every season) made a stink about mid-majors being over-valued. I liked the change, because seeing those of Herb’s ilk get rewarded for scheduling Arkansas Pine-Bluff 15 times and then squeezing out a couple of ‘quality’ wins in conference was really tiresome.
As you can probably tell, I really don’t have a problem with the way selections are handled currently. If the values used for selection purposes change that’s fine, but you can still game the system to an extent. No matter what is used, imo it should value the ‘little guys’ on par (as possible) with the power conferences.
I just posted a response that I’m not seeing. It had a single link in it, but it is also the size of a small atlas, so I think it probably also got caught in the spam filter.
I just posted a response that I’m not seeing. It had a single link in it, but it is also the size of a small atlas, so I think it probably also got caught in the spam filter.
Just woke-up myself 🙂
Sorry, I do not see your post … it could have been rejected for any number of reasons that I haven’t completely figured out.
Keep trying, and I’ll keep checking the filter. If you’re unable to get it to work, feel free to e-mail me @ wufpack76 @ hotmail .com and I will post it for you if you wish.
I haven’t gone into extreme detail on other rating systems mostly because this isn’t a controversial point I’m making. Every college basketball analyst that has any measure of statistical knowledge knows that RPI is a bad system. It has been almost universally derided as outdated and obsolete, with seemingly the only holdout being the NCAA itself.
So, here we go.
1) What are we actually trying to measure here? We’re trying to figure out which teams are the best teams, right? Or we can call it “which teams have played the best this season,” but that’s really the same question, because all we can actually see and measure is how well a team has actually played. I’ll quote again from the Nate Silver article I linked to earlier:
But there was a fundamental tension in the process: were we supposed to be picking the best teams or the most deserving ones? One example of the distinction is Xavier: did it deserve some consideration because a number of its players were suspended after a midseason brawl with Cincinnati? Xavier was probably more talented than its record at the time and might beat another potential 12th seed head to head. And yet, it was hard to call Xavier deserving; its problems were of the players’ making.
When I posed this question to David Worlock, the associate director for the men’s tournament, his answer was unambiguous: we were supposed to be picking the best teams. The committee members spoke frequently of the “shirts and skins test”: who would beat whom if they actually played a game?
According to the NCAA itself, we’re trying to find the best teams. That’s the idea. Now, if I give you no other information than “RPI tells me that team X is a better team than team Y” and then ask you who you would pick to win the game, in lieu of any other information, you would pick team X. That’s the whole idea of a rating system. Now, team X is not always going to beat team Y. There may be injuries, there may be matchup problems, one team may play out of their minds, one team may have an off night. However, if we check it over the course of hundreds of games or thousands of games, we expect that if RPI is any good at telling me which team is better, than in the aggregate I’ll win my picks more times than I lose. If not, then I’d conclude RPI doesn’t do a very good job of telling me which team is better. Can we agree on that? That’s how we evaluate models, by testing how well they represent what actually happens.
2) Margin of victory (and to clear up confusion, when I say “margin of victory”, I mean the average difference between your points scored and your opponents’ points scored. You don’t have a separate margin of victory and margin of defeat, we just average it all together and call it margin of defeat. So if your opponents have scored more than you on average, you have a negative margin of victory) is a better predictor of future success than W/L record. This is not up for debate. It’s been proven for basketball, baseball and football (and probably other sports I don’t care about) at all levels of competition. There is undoubtedly lots of writing on this that you could find, but I’ll just point to this post by Ken Pomeroy, which studies head to head matchups of conference teams. Just for example, this study shows that if you win your first home game by 5 points, on average you’ve got about a 35% chance to win the second road game. If you win by 20, you’ll win that second game closer to 60% of the time (There’s some nuance in there that Pomeroy discusses having to do with the fact that these are conference games, but the general gist is clear, MoV gives you significantly more information than W/L record). Now, referring back to point 1, something that has better predictive power is a better representation of how good a team actually is. MoV tells me much more about how good my team actually is than its record does.
3) One of the main issues with RPI is that it doesn’t use margin of victory (there are other issues, but that’s probably the biggest one). I’m not privy to the exact formulas behind BPI/Kenpom/Sagarin, but I know the general gist, and they all are based in some part on margin of victory (or, more exactly, offensive and defensive efficiency, which controls for pace). This is a better metric to use than just W/L, and simply by not using it RPI immediately is at a disadvantage. It’s not as good at figuring out how good teams actually are, so the way it calculates strength of schedule is inherently flawed. All of the other systems account not just for margin of victory, but also for strength of opponents, and they do it in much more effective ways than RPI does. Again, I don’t know exactly what the calculations are for opponent adjustment in each system, but I know that they’re done on a game-by-game basis (i.e. this game is weighted X because the opponent is ranked X is the system), rather than arbitrarily deciding that SOS is 75% of a team’s value. And that’s the thing, these systems are made by statisticians who know basketball and who know math, and they are models based on solid empirical evidence (does the model match what we see in real life?) and theoretical justification (does it make sense mathematically?). RPI is divided the way it is because it’s easy to divide by 4. RPI was built for simplicity, not for accuracy.
ESPN’s BPI, by the way, lessens the impact of blowouts and accounts for when key players miss games.
4) All these systems account for home/road results better than RPI does. For example, lets look at our upcoming game with UNC according to the KenPom ranking. The KenPom system sees that UNC is a better team than us, and estimates that, including the home court advantage (~4 points from a neutral floor on average), UNC should expect to beat us by 9. If we beat UNC, or even just lose to them by 1, the system will go “hey, they’re better than I thought they were”, and our ranking will go up. There’s a definite expectation for that game, a definite weight that it carries in our schedule. In RPI, it says “this game is worth 60% of a game if you lose and 140% of a game if you win”. So lets say we’re down by 1 at the end of the game, and Ralston gets a wide open look to win it at the buzzer. Lets be charitable and say that he’s a 50% shooter if you give him a good look at the basket. Essentially, we have flipped a coin to determine if the game is worth twice as much to our rating as it otherwise would be. Compare that to the KenPom method, which acknowledges that those two points are important, but also knows that whether or not that shot goes down is not indicative of a huge swing in the actual quality of the team. The RPI adjustment is better than nothing, but it’s still not good. It’s also inconsistent. If home/away matters for the quality of my wins, it should matter for the quality of my opponents’ wins too in calculating my strength of schedule, but the home/away adjustment is not used in calculating the strength of schedule portion of the RPI (presumably because they realized this would potentially set up situations where it is actually preferable for your RPI for you to lose to a good team at home than to beat them, which kind of shows why the whole thing is flawed in concept).
Well, I got most of it to post by breaking it up, I can’t seem to get the last numbered point to post, but here’s the final thing:
Look, I know that this is a really long post, and I don’t expect most of you to read it all, but you asked for reasons, and I’m giving them as clearly as I can. I’m not trying to be a know-it-all, argumentative, or proving that I can win the silly internet debate. I love basketball, and as the many post-game analysis posts I’ve made will attest, I love diving into the nitty gritty, eye-test details of not numbers but what is actually going on on the floor. But I also got a statistics degree from this great university that we all love, and I love talking about and explaining sports statistics. So I hope this post doesn’t come off as condescending or aggressive, because that’s not the spirit with which it was intended.
I’m not trying to be a know-it-all, argumentative,
^This has been an interesting and fun debate, imo.
Wulfpup – trying to understand the logic here. You get tired of the sendek’s and Greenberg’s gaming the system by going 9-7 in a power conference and beating up on the Arkansas pine-bluffs of the world ooc. You’d rather see them replaced by a team that goes12-4 in conference by beating the Arkansas pine-bluffs of the world but not winning their conference.
All I can say is that snow/ice days make for great posting.
Pup, Phoenix, Va – great stuff, fun reading!! (I would have used caps (certainly cap worthy), but as noted above, it might have been filtered)That’s an interesting distinction between “good” and “deserving”. MoV vs W/L record is also interesting.
First of all, let’s dismiss SMU from our discussion. Based on a number of cases going back to the Herbble, SMU was penalized by the Selection Committee for their OOC schedule, not just their wins or losses. If their OOC SOS was 200 instead of 300+, they would have received an at-large bid with the exact same record. If they had done better in their conference tourney, they would have been in (See the Herbble years for proof.) Of course that is a conclusion and not a fact…but when it predictably happens year after year it is close enough to a fact for me.
Quoting the Selection Committee doesn’t prove much because they so often contradict themselves when you pile up comments from different years. For me, what they do is more important than what they say.
I’m not convinced that MoV should be used to evaluate the 2nd or 3rd place team in a low or mid-major versus a team in the middle of a large, power conference. And to my mind this is the key issue that the Selection Committee has to wrestle with.
The bottom 150-200 teams are quite easy to eliminate. The conference champs and the top 20 or so at-large bids are easy. It’s the last 10 or so at-large bids that cause all of the hand-wringing…both on the committee and the bracketologists.
It’s clear that the Selection Committee has decided that it is going to pick “deserving” teams based on actual victories over good teams. The Selection Committee also has decided that it is going to penalize marginal teams that try to rack up a large number of wins against a weak schedule…and that penalty has been used on both mid-majors and teams from the power conferences. I see nothing wrong with this process; though I may be biased because State has fallen on the right side of the bubble so many times.
The selection process is fairly well understood by people like Lunardi and Palm and I think that we do a pretty good job around here. IF we can figure out, then the coaches and athletic directors pretty well know what is expected as well. So when they schedule like Greenburg, Herb, and SMU last year…then they know the risks they are taking and will have to live with the results.
Side Note:
The main issue with the process that I use is the assumption that a fixed measuring stick can be used year after year to evaluate the ACC. I use that simplistic assumption because I’m not going to go analyze 100+ teams until I’m getting paid (make that very-well paid) to do it.One other point is that the Selection Committee is fairly consistent from year to year. This conclusion is supported by the stats behind the Dance Card. So even if you don’t like the process, you ignore it at your own risk.
Packfanistan,
Before the gentler kinder Rick, you would have thought it was me
Nah, you were responsible for the comments that posted successfully and then later disappeared. 😉
I see only two ways for this team to get to the NCAAT: win out the regular season plus at least two in the ACCT, or simply win the ACCT.
It’s not any more complicated than that. We’ve lost too many games at this point, and too many of those have been truly bad ones, most of which came at home, where they shouldn’t.
Vawolf – I agree that while you can argue about whether RPI should so highly factor into decisioning, the committees actions are consistent and predictably.
Wulfpup – trying to understand the logic here. You get tired of the sendek’s and Greenberg’s gaming the system by going 9-7 in a power conference and beating up on the Arkansas pine-bluffs of the world ooc. You’d rather see them replaced by a team that goes12-4 in conference by beating the Arkansas pine-bluffs of the world but not winning their conference.
^Not exactly. This goes back to the “We’re a good mid-major but no one will play us” refrain.
Consider a conference like the Missouri Valley. Barring a complete collapse by either team, the Valley will get two bids this season: Northern Iowa and Wichita State. The Valley will receive a third team should both Wichita State and UNI be upset in the Valley tournament. I’m completely fine with this.
If you look at Wichita State’s and No. Iowa’s schedules, you’re probably not overwhelmed. They’re not terrible schedules, but they’re not the toughest ever, either. Aside from seeing them in person and on tv, how do you rate them and judge their results in the selection process? In the past, you could clearly see that these type teams were good teams and worthy of inclusion – but their schedules and numbers could often be used against them for exclusion prior to the latest RPI tweak. This season, Wichita is currently RPI 17 and UNI is RPI 19.
Back in the day, there was a legitimate complaint from mid-major schools and conferences that they couldn’t get power conference schools scheduled – even on the road. The bottom of mid-major conferences would be such that their numbers would become an albatross for any team at the top of the conference that could possibly be considered at-large worthy. So, the legit question was: “What exactly do you want us to do if we can’t get on any good teams’ schedules?”
Consider that in the past, it would be the ‘norm’ for either Wichita State or UNI to be excluded once they lost in their conference tourny. They would be excluded in favor of a power conference team like Georgia – a team that accomplished nothing of note in non-con – but hey, their RPI is decent and they get brownie-points for being in the SEC and playing Kentucky twice. A team that lost to Georgia Tech (bottom of the ACC), Minnesota (mid to bottom of the B1G), got swept by South Carolina (yuck), and lost at home to Auburn (vomit) is more worthy than either Wichita State or UNI?
No way. A team like Georgia gets every opportunity in the world to prove their merit and can schedule accordingly. In the past, coaches like HWSNBN and Greenberg would rely on merely being in a ‘power conference’ to do their work for them. They wouldn’t dare take the chance of having a Wichita State or UNI kick their ass in non-con – instead they’d hedge their bets, never leave home and ‘test’ themselves against Arkansas Pine-Bluff. Then they’d whine about how “tough” it is in a power conference and how merely facing UNx or Duke twice is enough to prove their worth.
The NCAA and selection committees saying “Yeah, um – NOPE.” is absolutely fine by me. Earn your freaking pay check, and earn your way in to the Tournament. Is it really that much to ask to at least attempt to challenge yourself and take a cue from coaches like Gottfried?
For all the faults the RPI may have, everyone knows the rules of the game and what they need to do. If they choose not to adhere to how selections are made it is their choice. It bit SMU last season. It bit VT under Greenberg in seasons past. C’est la vie.
I’m not convinced that MoV should be used to evaluate the 2nd or 3rd place team in a low or mid-major versus a team in the middle of a large, power conference. And to my mind this is the key issue that the Selection Committee has to wrestle with.
^This. 1000 x this.
13OT – While I agree with the emotions, looking through the committee’s eyes, we only have one loss outside the RBI Top 100. And that was the WF debacle.
According to the rankings at the time the games were played, Purdue, WVU, Cincy, UVA (2), UNC and ND would all have been upsets. (Purdue has since slumped)
So that leaves four losses to make us mad. Wofford (by 1), Miami (by 5), Clemson (by 11) and WF (by 4). Wofford is apparently pretty good, the last time I looked, their RPI was 49 – ours is 43. But those other three are the ones that keep me awake at night.
Give us those three and we are at 18/8 and 9/4 and pretty comfortable. Makes me crazy!
I was trying to be logical instead of emotional. We’ve simply racked up too many losses this season, and all I’m saying is that if we don’t get at least 20 or 21 wins, it’s going to be the NIT.
I have no doubts whatsoever that the Wolfpack is capable of beating anyone in the country, but they’ve simply got to stop the losing and start winning. We haven’t won 2 straight games since December 12.
13ot – none of the indicators anyone uses back up your claims. The conventional wisdom is that if we go 4-1 and win 1 game or 3-2 and win 2 games, we’re in. That’s not guaranteed but unless the committee uses a different criteria than they have in the past…
^4-1 likely gets us in even w/ an opening game loss @ the ACC tournament … but we shouldn’t chance it if State can go 4-1 down the stretch 🙂
Yea wulfpup – I agree, but we’d have another bad loss if that happened and it could hurt our RPI enough to get us in the danger zone.
Interesting discussion about the selection process. From my observations VAwolf’s thought process is pretty straight on. Somehow, the selection committee has determined that it is their job to punish poor OOC scheduling. Hence, possibly deserving teams like SMU, Clemson last year Seth’s VT teams are left on the outside. A premium is also placed “Marquee” wins over consistency. A team that comes up big a few times then boots a few is given precedence over a team that just wins the ones they should. I guess this could be justified as selecting the team with the opportunity to pull a big upset or make a deeper run.
IMO the Pack needs a minimum of 4 more victories to get in the NCAA tourney. Any 4 of the last 5 regular season games should get us us. 3 will get us in if Cheater Hill is one of them and we win the opening game in the ACCT. 3 without a win against the Cheats probably means we need 2 in the ACCT to really feel safe. 3-2 to finish the reg season, no win at UNC, and a win against a bottom feeder on Wednesday of the ACCT leaves us right at the last team in/last line…probably out.
- You must be logged in to reply to this topic.
