Friday, August 03, 2012

Most accurate fantasy football experts, 2008-2011

Greetings. As longtime readers are well aware, since 2007 I've been running a contest that analyzes the accuracy of fantasy football experts' preseason rankings. I look at rankings for each of the offensive skill positions and generate results for each position as well as an overall accuracy winner across all four positions. You can find the details behind how the annual rankings are created here with the 2011 results as posted to the NY Times' Fifth Down blog back in January.

If you're uninterested in the details of the contest and just want the results, skip ahead through the next few paragraphs to the charts below. Otherwise, read on.

We all know that accuracy results from one single year are great, but obviously when you combine several years you get a better picture of which sites really excel. I ended up throwing out the 2007 results I had since I ran that contest by merely looking at sites' overall rankings; in future years I switched to looking at specific positional rankings. Over the course of the contest from 2008-2011, I have amassed submissions from 79 different sites/experts. I decided that it's really not fair to combine the sites that have participated for just 1 year with those that have been in it for multiple years; too much can happen in 1 year, good or bad. So I threw out all sites that only participated once, which amounted to eliminating 33 sites (note: this includes Fantasy Shrink, the 2011 winner, though it also includes a few sites that aren't even in business any longer).  For those that are curious, 17 sites have participated for 2 years, 12 sites have participated for 3 years, and 16 sites have participated all 4 years.

In creating the original rankings I developed both a score (essentially the accumulation of points from all players in a position by measuring the difference between the expert's preseason ranking for a player and the actual ranking of that player at the end of the fantasy season) and a ranking (based on the score relative to other participants) within position. Now in order to get at the multi-year results, because the scales differ from year to year, I normalized the scores for each year where 0 is the average expert, negative scores are more accurate than the average expert, and positive scores are less accurate than the average expert. I then averaged across the number of years that that expert had participated. I thought about creating the multi-year results by merely averaging the rankings but decided that the scores are a more appropriate way of analyzing sites since it rewards the more accurate sites and punishes those who were just way, way off.

What you have below is a list of only the top sites that have participated at least twice. Each year when I ask for participants in this contest I promise that only a top percentage will be displayed to the public. I do this largely to keep sites participating but also because I'm really not in the business of spreading bad news about sites. I completely understand your desire to see the full list but hope this abbreviated version will do.

Again remember that due to the normalization I did, negative scores indicate more accurate predictions relative to the average expert. So here, negative = good.

First up, the overall accuracy across all 4 positions from 2008-2011:



Next, quarterback accuracy from 2008-2011:

 

Running back accuracy from 2008-2011:

 

Wide Receiver accuracy from 2008-2011:


Finally, Tight End accuracy from 2008-2011:


Congrats to the crew of fantasy writers/rankers at ProFootballWeekly.com!

Update: as requested (and sorry I didn't think of this originally) here are the 46 sites that participated for 2+ years and ended up being involved in the analysis. They are listed below alphabetically:

4for4.com
Accuscore
Ask the Commish
Bruno Boys
CBSSports.com, Dave Richrd
CBSSports.com, Jamey Eisenberg
Draft Analyzer Ultimate
Draft Sharks
ESPN
Fantasy Football Geek Blog
Fantasy Football Nerd
Fantasy Football Starters
Fantasy Hulks
Fantasy Mojo
FantasyFootball.com
FantasySharks.com
FantasySherpa
Fantazzle
Fantistics
FFCalculator (ADP)
FFManiaxs
FFToday.com
FFToolbox.com
FFTrader
FFXtreme
Football Outsiders
Footballguys
FOXSports
Hatty Waiver Wire Guru
Junkyard Jake
KFFL
LestersLegends.com
NFL.com
Pigskin Addiction
ProFootballFocus
ProFootballWeekly.com
Pyromaniacs
RapidDraft
Real Time Fantasy Sports
Rotoprofessor
Rotoworld
Sports Illustrated
The Huddle
WhatIfSports
Yahoo!

6 comments:

Anonymous said...

Can we get a list of the 46 experts who were analyzed? I like seeing teh winner, but also want to see if, for example, Andy Behrens was eligible?

Anonymous said...

For some reason, I cannot see anything in the rankings boxes. Can you advise or change it so its just a list and not an apparent image?

Thanks.

Anonymous said...

When you say Pro Football Weekly Fantasy is that Pat Fitzmaurice in particular or consensus? Same for FootballGuys, is that David Dodds or another individual or their consensus rankings? Putting in paranthensis who at each site (like you did with Dave Richard) would be very helpful. Thanks for all your hard work on this.

Anonymous said...

I can't see anything on the boxes either. If I click on them, they open a window with nothing in it.

Reid said...

I just want to thank you for this!
I'm sure it was a lot of work compiling and working with all this raw data. I just wanted to let you know how much I appreciate your work!
Cheers and good season!

Anonymous said...

I'd like to see Fantasy Football Champs (ffchamps.com) and John Hansen (Fantasyguru.com) get added to the list.