Greetings. As longtime readers are well aware, since 2007 I've been running a contest that analyzes the accuracy of fantasy football experts' preseason rankings. I look at rankings for each of the offensive skill positions and generate results for each position as well as an overall accuracy winner across all four positions. You can find the details behind how the annual rankings are created here with the 2011 results as posted to the NY Times' Fifth Down blog back in January.
If you're uninterested in the details of the contest and just want the results, skip ahead through the next few paragraphs to the charts below. Otherwise, read on.
We all know that accuracy results from one single year are great, but obviously when you combine several years you get a better picture of which sites really excel. I ended up throwing out the 2007 results I had since I ran that contest by merely looking at sites' overall rankings; in future years I switched to looking at specific positional rankings. Over the course of the contest from 2008-2011, I have amassed submissions from 79 different sites/experts. I decided that it's really not fair to combine the sites that have participated for just 1 year with those that have been in it for multiple years; too much can happen in 1 year, good or bad. So I threw out all sites that only participated once, which amounted to eliminating 33 sites (note: this includes Fantasy Shrink, the 2011 winner, though it also includes a few sites that aren't even in business any longer). For those that are curious, 17 sites have participated for 2 years, 12 sites have participated for 3 years, and 16 sites have participated all 4 years.
In creating the original rankings I developed both a score (essentially the accumulation of points from all players in a position by measuring the difference between the expert's preseason ranking for a player and the actual ranking of that player at the end of the fantasy season) and a ranking (based on the score relative to other participants) within position. Now in order to get at the multi-year results, because the scales differ from year to year, I normalized the scores for each year where 0 is the average expert, negative scores are more accurate than the average expert, and positive scores are less accurate than the average expert. I then averaged across the number of years that that expert had participated. I thought about creating the multi-year results by merely averaging the rankings but decided that the scores are a more appropriate way of analyzing sites since it rewards the more accurate sites and punishes those who were just way, way off.
What you have below is a list of only the top sites that have participated at least twice. Each year when I ask for participants in this contest I promise that only a top percentage will be displayed to the public. I do this largely to keep sites participating but also because I'm really not in the business of spreading bad news about sites. I completely understand your desire to see the full list but hope this abbreviated version will do.
Again remember that due to the normalization I did, negative scores indicate more accurate predictions relative to the average expert. So here, negative = good.
First up, the overall accuracy across all 4 positions from 2008-2011:
Update: as requested (and sorry I didn't think of this originally) here are the 46 sites that participated for 2+ years and ended up being involved in the analysis. They are listed below alphabetically:
Ask the Commish
CBSSports.com, Dave Richrd
CBSSports.com, Jamey Eisenberg
Draft Analyzer Ultimate
Fantasy Football Geek Blog
Fantasy Football Nerd
Fantasy Football Starters
Hatty Waiver Wire Guru
Real Time Fantasy Sports