Back in February, I posted an article to the NY Times pro football blog, The Fifth Down, about the accuracy of fantasy football experts when it comes to preseason overall rankings. Since then, I have tweaked my methodology for looking at the expert rankings and added a site (Rotoworld) to the analysis. And it's a good thing I added Rotoworld, too, since they led the way on almost every angle of this analysis. But before I get too deep into the results, I just want to put a big warning that these results have flaws; it is impossible (as far as I can tell!) to do a perfect analysis of this sort of thing since every site has a different methodology and likely different scoring systems that they're working off of. And perhaps more importantly, this is merely one year of analysis that I'm looking at; anyone who works with stats can tell you that one year of results doesn't hold very much weight...which is why I plan on continuing this (and hopefully adjusting my ranking process until I'm more comfortable with it) for years to come. Additionally, please keep in mind that I assume you have a brain, that you aren't going to just look at this list and never ever look at SI again; that would be ill-informed of you. This is just meant to be a supplementary tool as you start looking at various experts out there.
Also, there are some sites (like fantasyguru.com) that I only had positional rankings for, not overall top 100 rankings. This makes a difference only because I am attempting to compare apples to apples as much as possible. Perhaps this coming year I'll switch my analysis around by looking at these positional rankings the sites churn out rather than the overall Top 100 types of rankings (and pulling out the positional piece from the overall). See below for more on the positional issues that I've tried to address.
So I welcome your comments and if anyone has a better way of looking at this (keeping in mind my limitations of using just Excel and of course having a full-time job), by all means, please let me know.
So without further ado, here are the results:
1. Expert rankings from an overall perspective (see this article for an explanation on how I did these calculations):
1. Rotoworld (6005)
2. Ask The Commish (6,031)
3. FootballGuys.com (6,142)
4. FF Cheatsheets (6,157)
5. NFL.com (6,167)
6. FFToolbox (6,211)
7. CBS Sports (6,221)
8. Yahoo! Sports (6,253)
9. Fantasy Football Xtreme (6,301)
10. Fox Sports (6,345)
11. Pigskin Addiction (6,370)
12. SI (6,419)
13. ESPN (6,447)
14. ProFantasySports (6,557)
A few months ago many of you felt that looking at sites from an overall ranking and not a positional ranking was flawed (see the comments section of that article linked above for details). And I agree that positional analysis needs to be taken into account, but I am also very interested in how well a site can predict overall rankings as well. That still has value to me and in my book, it adds to what makes an expert an expert. But as requested, I took a stab at looking at the rankings within positions from the info in Top 100 rankings, so that I wasn't just looking at these from an overall rankings perspective. So this next list is an attempt to address some of the issues related to how overall rankings are created and taking positions into account a little more - but again, it's likely these results are flawed to some extent since I think that's a really tall order. But hey, read on anyway...
2. Expert rankings from a positional perspective (the number in parentheses is the total number of spots off a site's preseason ranking was from the final season outcome):
QB rankings
1. FootballGuys (192)
2. FFToolbox (196) and NFL.com (196)
4. Ask The Commish (197)
5. Rotoworld (200)
6. FF Cheatsheets (203)
7. Pigskin Addiction (211)
8. CBS Sportsline (213)
9. FoxSports/Rotter (215)
10. Yahoo! (221)
11. SI (225)
12. ESPN (229)
13. ProFantasySports (246)
14. FFXtreme (261)
RB rankings
1. Rotoworld (984)
2. FoxSports/Rotter (1000)
3. FF Cheatsheets (1007)
4. FFToolbox (1010)
5. Yahoo! (1013)
6. Pigskin Addiction (1020)
7. Football Guys (1021)
8. ESPN (1023) and NFL.com (1023)
10. ProFantasySports (1032)
11. SI (1037)
12 Ask The Commish (1039)
13. FFXtreme (1045)
14. CBS Sportsline (1060)
WR rankings
1. Rotoworld (1075)
2. FFXtreme (1078)
3. FF Cheatsheets (1082)
4. CBS Sportsline (1122)
5. Football Guys (1124)
6. FFToolbox (1127)
7. Pigskin Addiction (1150)
8. ESPN (1160) and Yahoo! (1160)
10. FoxSports/Rotter (1170)
11. ProFantasySports (1182)
12. Ask the Commish (1201)
13. SI (1237)
14. NFL.com (1245)
TE rankings
1. Football Guys (94)
2. NFL.com (95)
3. Rotoworld (96) and Yahoo! (96)
5. Ask the Commish (98)
6. FFXtreme (99) and FoxSports/Rotter (99)
8. Pigskin Addiction (100)
9. CBS Sportsline (104) and FF Cheatsheets (104) and FFToolbox (104)
12. ProFantasySports (105)
13. ESPN (107) and SI (107)
Now if you take those numbers and just do a quick averaging of the positional rankings (so for Football Guys, I'd average together their rankings of and 1 (QB), 7 (RB), 5 (WR), and 1 (TE) for a positional average of 3.5), you get the following:
1. Rotoworld (avg 2.5)
2. Football Guys (avg 3.5)
3. FF Cheatsheets (avg 5.25) tied with FF Toolbox (avg 5.25)
5. NFL.com (avg 6.5) tied with Yahoo! (avg 6.5)
7. FoxSports/Rotter (avg 6.75)
8. Pigskin Addiction (avg 7)
9. Ask the Commish (avg 8.25)
10. CBS Sportsline (avg 8.75) tied with FFXtreme (avg 8.75)
12. ESPN (avg 10.25)
13. ProFantasySports (avg 11.5)
14. SI (avg 12)
One final note, after a lot of reflection I did not include fan favorite Football Outsiders in this analysis since they have such a unique way of looking at rankings (which I love, by the way) - additionally, their rankings can be adjusted based on the scoring system you use, which is awesome, but it therefore means they don't really have one overall top 100 ranking; they have many, many overall rankings you can choose from. So it's unclear how they did in comparison; I can certainly pull out some of the players that FO did a good or bad job predicting though, so if that's of interest, let me know.
If you want all the gory details, see this Google spreadsheet. If you have trouble with that spreadsheet though and just want me to e-mail you the excel version, I can do that, too. Just contact me.
Enjoy!