Chaos Rankings

Tuesday, August 18, 2009

Do Preseason Rankings Have Any Merit?

I've been putting off doing my preseason rankings this year, mostly out of laziness. It requires a lot of work to gather all the information about 120 teams' returning coaches, coordinators, starters, and special teams. So in lieu of actually doing all the fact-finding, I decided to look over the data I already had and see what it's telling me about whether or not my preseason ranking actually have any bearing on who comes out on top at the end of the year. As I've stated before, most preseason rankings are just throwing darts at a team list and then applying some subjective filtering based on historical performance (effectively voting for laundry) or "gut feelings" about which team is poised to have a huge year. Case in point: Sports Illustrated lists Utah at 19 for 2009-10, despite having finished with an undefeated record last year and returning half their starters and their coach. Meanwhile, the Alabama team that Utah throttled in the Sugar Bowl comes in at #8 with virtually identical circumstances.
Anyway, if my preseason rankings aren't indicative of future performance, then they're kind of worthless, even if based on based solely on measurable data rather than intuition. What follows is a table showing the preseason and final rankings for each team that appeared in the top 25 spots on Chaos Rankings top 25 in either of the past two seasons.

07-08 Pre
07-08 Final
08-09 Pre
08-09 Final
Arizona State59161676
Ball State85514711
Boise State114116
Boston College28131333
Notre Dame2510210956
Ohio State37413
Oklahoma State43535123
Oregon State21303225
Penn State2322228
South Florida29201843
Texas A&M22556093
Texas Tech41212110
Virginia Tech16111222
Wake Forest18333036
West Virginia431027

Of this group of 44 teams (more than 1/3 of Div. I-A for those keeping score at home), there were only 14 moves of 5 or fewer spots (from 88 slots). And then 30 moves of 10 spots or less, and 39 moves of 25 spots or more (enough to drop any team out of the top tier). That's a lot of movement in what's supposed to be the upper crust. If you look at the entire Div. I-A data set, for both seasons, about 50% of teams moved at at least 20 spots up or down from their preseason rankings. In fact, almost 1/3 of teams moved 30 or more spots.
If you're curious, the most consistent teams are, in order: Army, USC, New Mexico State, Oklahoma, BYU, Ohio State, Oregon State, Virginia Tech, Arkansas State, and Boise State. Those are your teams that perform to expectations, whether good (USC, Oklahoma, etc) or bad (Army, Arkansas State, etc). The biggest movers and shakers are, in order: Rice, UCF, Michigan, Notre Dame, Illinois, Ole Miss, UConn, Minnesota, Mississippi State, and Western Kentucky. Next on the list would be Kansas. Those are your teams that had a very down year or a very up year and skewed either their preseason ranking or their final ranking considerably.
In 2007-08, only 3 of the final top 10 came from outside the preseason top 15, which so isn't bad. Conversely, 3 preseason top 10 teams aren't represented in the final top 25, which is a pretty lousy prediction. In 2008-09, things get worse. 6 of the final top 10 came from outside the preseason top 15, 5 from outside the top 20, and 2 from outside the top 25, while again 3 preseason top 10 teams weren't in the final 25. I'll grant the occasional team experiencing a total collapse (Wisconsin, Notre Dame, Michigan) or coming from nowhere (Kansas, Mizzou) or even mid-major yo-yoing (TCU, Utah), but more the most part, teams expected to finish in the top 10 shouldn't be finishing outside of the Top 25. And I would expect to see nearly all of the final top 10 coming from at least the preseason top 25.
So, what does this all mean? It means despite my best ranking-design efforts, there's a lot of movement going on from the preseason to the final rankings, which leads me to believe that the preseason rankings aren't doing a very good job of predicting anything. So I'm going to forgo this year's preseason rankings, at least for now. In the mean time, I'd like to go back and try to correlate some data and see if I can pick out any patterns that indicate future performance based on available preseason data.


Post a Comment

<< Home