Skip to main content

How Accurate Were They in 2006?

A great question was brought up by Aaron in a recent comment about my Average Ranking Positions post. Aaron asked if I had any of the Top 100 lists from this same group of experts from 2006 and if so, how accurate the rankings were - so I've created a new spreadsheet, a 2006 Predicted Rankings vs Actual Fantasy Results spreadsheet. To find all of these 2006 rankings, I'm using the Internet Archive Wayback Machine, a website that has archived much of the Internet. Type in the website address you're interested and voila, a dynamic November 2005 version of that page is at your fingertips (or whatever other date you're looking for). Pretty cool, huh?

So I found ten of the same rankings I used this year for 2006 - of the ones I couldn't find, FF Toolbox had only the top 60 for 2006 so I had to throw it out; NFL.com didn't provide an 2006 overall ranking, just positional; and Fantasy Football Xtreme's 2006 version was not free. Otherwise, a ranking from each of the other sites was acquired and I did the same averaging thing for these rankings as I did for the 2007 rankings. Then I compiled the fantasy point totals for each of these players based on the standard scoring of one of my leagues (players who have retired or are not on active rosters were not included, like Tiki or Corey Dillon). The scoring system makes all the difference here, so the scoring rules I derived my fantasy points from are:

Passing Yards (25 yards per point)
Passing Touchdowns (6)
Interceptions (-2)
Rushing Yards (10 yards per point)
Rushing Touchdowns (6)
Reception Yards (10 yards per point)
Reception Touchdowns (6)
2-Point Conversions (2)
Fumbles Lost (-2)
Offensive Fumble Return TD (6)

Once I had all of the fantasy point totals for the season, I ranked all of the players based on fantasy points scored in the season. I compared this ranking of fantasy points scored to the 2006 pre-season rankings the experts provided. Then I took the absolute value of the difference between the two for each player and summed it up for each ranking. So the ranking that is the most accurate should have the smallest number. See the fuzzy chart below - I know it's hard to read the ranking names along the x axis, but the ranking with the smallest number, or best accuracy, was SI. So in order of greatest accuracy, the list goes:
SI
CBS Sportsline
ESPN
FootballGuys.com
Fox Sports
Pigskin Addiction
Ask The Commish
ProFantasySports.com
Yahoo Sports
FF Cheat Sheets

If you look at the spreadsheet (or above hard-to-read chart) you can see that ultimately they are all really close together - the span is not nearly as great as I thought it might be. Nor do any of them really seem to be all that accurate. This is also just one year's worth of data, so perhaps when time permits I will look into finding 2005 and 2004 data to solidify these findings.

I found it interesting to see which players had the most errant and accurate predictions from the expert rankings. Most errant included Jerry Porter, Chris Perry, Chris Brown, and Koren Robinson. The most accurate rankings were (not surprisingly) for LT, Larry Johnson, Reggie Wayne, and Steven Jackson.

Hope that was enlightening for some of you and I promise to provide posts with more timely fantasy information soon.



Comments

Anonymous said…
I always wondered about that! Awesome post. I wonder where the Bruno Boys rank?
Corey/FFGoat, I'll try to find the Bruno Boys 2006 ranking - if you already have it, send the URL my way, I'll add it to the spreadsheet and let you know the outcome. In fact if anyone else has a 2006 ranking they're curious about, just send me the ranking or URL and I will put it in the spreadsheet.
Unknown said…
This is great; thanks! I'm thinking of crunching the numbers a little differently. It would be interesting to see if experts vary across positions, like does SI do a better job at ranking qbs or rbs? Which expert is the best at picking wrs? I also was planning on throwing in a term that so misrankings the best players matters most. My intuition is that it's a lot worse to pick the second best running back as the first, rather than to rank the 12th best running back as the 11th (the difference in fantasy points/game between LT and Larry Johnson last year was 20%, between Rudi Johnson and Ahman Green: 2.3%). Ultimately, it would be nice to have a weighting function that punished in direct proportion to the amount of fantasy points the misranking cost you, but I probably won't get around to that. Whatever I do get around to, I'll post here.
Anonymous said…
Any chance you or a reader has access to last year's Football Outsiders' rankings? They're likely the most scientific of anything out there.

Does anyone have links to any other attempts to judge football projections? (Baseball projections is a big interest of mine.)
Unknown said…
I ran the numbers to give separate rankings for qb and rb. I also applied a linear weighting as mentioned above (it is more important to rank the #1 player correctly than the #10 player). Technically speaking, for each expert I took the top n players for each position, and compared them against the expert, such that
s= 1/r_max * Sigma [(r_max + 1 - r_a)/r_max * (1 - |r_a - r_p|/(r_max - 1)]
where r_max is the highest examined ranking (top 10 qbs, top 20 rbs), r_a is the actual rank based on yahoo point total for that player, and r_p is the expert's predicted rank for that player. If the expert didn't include a player ranked player with his top n players for that position, they received a score of zero for that prediction. This score is an ad hoc measurement of agreement score, since I did want to include the weighting.

For running backs, the best experts were (scores are normalized so 1.0 is perfect ranking):
1. ask the commish(.4107), 2. fantasy football cheat sheet(.4043), 3. football guys(.3786), 4. yahoo(.3655), 5. pro fantasy sports(.3720), 6.si(.3595), 7. fox(.3583), 8.cbs(.3506), 9. pigskin addiction (.3400), 10. espn(.3263)

For qbs:
1. si(.5980), 2. ask the commish(.5798), 3. yahoo(.5535), 4. espn(.5535), 5. football guys(.5434), 6. fox(.4989), 7. cbs(.4949),8. pro fantasy sports(.4829), 9. pigskin addiction (.4284), 10. ff cheat sheet(.4102)

Given one sample year, we really don't have any statistical power to say anything, but this is a start. And on a less technical note, let me just point out that none of the experts properly ranked LT #1, which was huge given the 97 point differential between him and Larry Johnson.
Aaron, this is awesome. Do you mind if I put into a post where it might have more visibility?

Quick question - WRs? Any reason you chose not to do them other than lack of time?

I had also noticed that no one ranked LT as #1 last year - stunning, isn't it? So whose to say that Frank Gore or Steven Jackson won't outscore LT this year. In fact, I've read a few things recently about LT's decline - or perhaps his return to normalcy is a better way of phrasing it. Interesting stuff.
Unknown said…
Hey Librarian,
I realized last night I actually calculated the weighting factor slightly different then I described above. Here are the scores for the method described in the post:

WR:yahoo(.6036), football guys(.5929), FF cheat sheet(0.5792), espn(0.5750), fox(0.5679), Ask the commmish(0.5465), cbs(0.5447), si(0.5387), pfs(0.5346), pigskin(0.5310)

QB:
si(.4889), commish(.4788), football guys(.4707), pfs(.4586), yahoo, (.4485), espn(.4404), cbs(.4222), fox(.3939), ffcs(.3899), pigskin(.3636)

RB:
commish(.5072), football guys(0.4947), pfs(0.4875), pigskin(.4453), yahoo(0.4417), ffcs(.4393), si(.4232), cbs(0.4066), fox(.3798), espn(.3667)

If you can arrange these posts into something coherent, you are more than welcome to do so. Good luck with your draft.
Is not it amazing - one last year I realized that was # 1 and LT? So Frank Gore or Steven Jackson this year, not outscore LT. Or maybe a better way to ruin a normal return groan choose the thug - in fact, I read something recently about the decline of the LT. Interesting stuff.
Unknown said…
Excellent blog you’ve got here.It’s difficult to find high-quality writing like yours nowadays. I really appreciate individuals like you! Take care!! Please check out my site.
Daily Fantasy Sports Strategy

Popular posts from this blog

Back on the fantasy wagon

With my library conferences behind me (for a few weeks at least), I can now concentrate on fantasy football again. If you happen to be a librarian and are interested in how my co-presenter and I tied together fantasy football and information literacy, check out my co-presenter’s blog . Though I imagine most of you would much rather catch up on the following football news and thoughts for the week: I remain a bit dazed and confused by my certainty last summer that Cedric Benson was going to rock my RB2 world in 2007. Didn’t happen and now he’s moved onto ankle surgery, alleged intoxication and resisting arrest on boats. Awesome. You can practically taste Matt Forte’s value skyrocketing here. In fact, wondering about rookie RBs in general? Well then check out FF Xtreme’s The Day After: Ranking the Rookie Running Backs . I love reading articles or stats that challenge my preconceived notions – The Fifth Down’s Chad the Completion Percentage Machine does just that. No offense to thos

Streaming Defenses - Week 16

Don't know about you all, but in our house this week is just chock full of events, visiting with family, and getting organized for our upcoming time off over the holidays. Lots of scrambling going on around here and attempts to install some sense of gratitude in my boys. What am I grateful for this season? Many things but let's start with Bob. Bob has been my fantasy football rock, coming back week after week to post. Hats off to you, Bob, and thank you for your diligent work and research. So here he is with this year's final streaming defense post.  Well it’s the last week of the Fantasy Football Playoffs in most leagues, and my last post of the season. I hope you all found these posts useful. And if I failed you, you have to murder a lot more experts before you get to me. I myself have had a terrible playoff season. I’m out of all but one of my leagues due to the weird games the last two weeks. Between weather that didn’t materialize to normally consistent players co

Start/Sit Week 4 and a smattering of rankings

It's been awhile, my fantasy football friends. I have lofty intentions of getting several posts up here each week, but life is busy and I will do my best. So in the intervening 6 years since I've posted regularly on here, our family added another child (now there are 3!), moved back to Boulder for a sabbatical year and then returned to Knoxville, and we have basically been chugging along like many of the rest of you. My two boys (who turn 12 and 9 soon) are in a fantasy football league for the first time this year with their buddies, which ostensibly is the just the dads (and mom!) playing one another of course. But my oldest is taking it super seriously and what kind of Fantasy Football Librarian would I be if I didn't step up my game for him? So I'm back - and am interested in the fantasy landscape - lots of the same awesome sites out there, but many new ones as well. I'll try to incorporate a mix. Please be sure to let me know if my 2006 ways aren't working i