Skip to main content

Multi-year results for accuracy with QB rankings

Happy summer! To celebrate the new season and my return to a land where I can get e-mail and cell phone reception (had a great time this weekend but I do love the creature comforts of civilization...), I have a little something fun for everyone: the start of the aggregate accuracy analysis results. For those who are new to the blog, I've spent the past several years analyzing the accuracy of preseason expert rankings to find out which sites did the best job at predicting position rankings for the season. I started in 2007 and looked at 14 websites; after joining up with the Fantasy Sports Trade Association the following year it grew to 24 websites, and then a whopping 48 sites in 2009. At this point it has become almost entirely a submission process so if your favorite site isn't included, it's likely because they either chose not to submit or just do not know about the contest. You can read all of the details about how I analyze the rankings each year here and if you're a site that's interested in participating this year, stay tuned for more info on that front in August. Also, I want to note that one important condition of participation is that I keep all of the results of the contest confidential except for the top performers, so I unfortunately can't share the full results of each year's contest with you all. Besides I'd really prefer to focus on those who performed well year in and year out versus those who didn't do such a strong job with their rankings.

So I've had a plan to combine the results of the contest each year to create a multi-year assessment of each site, which is far more informative than just one year of results, and I finally had the time to do the analysis. Today we will take a look at quarterback rankings to see which site has been the most accurate with their preseason QB rankings from 2007-2009. Over the past 3 years I've included 54 different sites in my analysis, however only 10 sites have been included all 3 years, and and additional 13 sites have been tracked for 2 years, so for the multi-year analysis I'm focusing on sites that have participated in at least 2 years of the contest. Honorable mentions for those that did well in their 1 year of participation are noted below, too, though.

As you can see above, the contest has grown enormously and with the variation in the number of sites ranked each year, I decided to calculate a percentile ranking for each site each year. For example, Football Guys had the best QB ranking of the 14 sites I analyzed in 2007 so they were in the 7th percentile (1/14) - keep in mind this is not like the SATs; in this case you actually want a low percentile. After calculating the percentile for each year, I averaged each site's ranking from each year it was included in the FFLibrarian accuracy contest. The lower the number in the average column, the better the site did compared to their peers. Ask The Commish's 0.30 percentile means they averaged being in the 30th percentile in QB pre-season rankings over the last three years - that's the best of any of the sites I have data on for all three years. For comparison's sake, the site with the worst percentile ranking for QBs was averaging 0.86 for all three years (0.81 for sites with 2 years of data; 0.98 for those with 1 year).

Sites that have 3 years of participation
Ranking Site Average
1 Ask The Commish 0.30
2 Rotoworld 0.31
3 0.38
4 ESPN 0.44
5 FFToolbox 0.45

Sites that have 2 years of participation
Ranking Site Average
1 Football Guys 0.16
2 Fantasy Hulks 0.27
3 KFFL 0.34
4 0.43
5 Fantazzle 0.46

Sites that have 1 year of participation
Ranking Site Average
1 WhatIfSports 0.04
2 Junkyard Jake 0.06
3 0.08

I'm sure there are important things I'm forgetting to mention or notes that I should be making here, so please feel free to ask questions or make comments. As always, this is a work in progress so if you have a better method, please let me know but in the meantime a big kudos to my husband for talking all of this out with me. Just one of the many times it pays to have fallen in love with an economist/stats dude.

I'll be posting the results for RBs, WRs, and TEs over the next week or so in addition to my normal linking. Enjoy and a hearty congrats to Ask the Commish!


davinci78 said…
Well done and thanks for taking the time.
davinci78 said…
This comment has been removed by the author.
HughesNet said…
Good read. Interesting to see Rotoworld and outpacing the big boys across the positions. Interested in seeing more.

Popular posts from this blog

Labor Day FF links

Happy long weekend! Here's hoping most of you are free to enjoy the 3-day weekend and the relaxation and fun that comes with it. I spent my Sunday participating in 3 fantasy football drafts scattered throughout the day and night, but I think I've finally wrapped up my drafts for the year. So like many of you I'm now turning my attention to the Week 1 games! But before we get to some player rankings and prep for Week 1, I want to make sure that all you experts out there are aware of the upcoming deadline for the annual accuracy contest that I run with the Fantasy Sports Trade Association. I compare preseason positional rankings from experts to the final outcome of the fantasy season to see which site had the most accurate preseason rankings, and this year's deadline is fast approaching: September 9th by kickoff. Check out the info on the FSTA site for more details and be sure to e-mail me your submissions. Now, onto Week 1! First up, let's get to some of this

2010 Week 2 consensus start/sit

Greetings and happy weekend. I've got this week's consensus start/sit for you which is a tally of all the start/sit votes that players received in the various start/sit articles that I linked to earlier in the week. So while these aren't my personal recommendations, I do think knowing what the consensus is could be of help when making lineup enjoy! Note: the players are listed in no particular order. QB Start: Michael Vick, Kyle Orton, Matt Ryan, Jason Campbell Sit: David Garrard, Alex Smith, Jay Cutler, Vince Young, Carson Palmer RB Start: LeSean McCoy, Darren McFadden, Ahmad Bradshaw, Brandon Jackson Sit: Tim Hightower, C.J. Spiller, Beanie Wells, Ronnie Brown WR Start: Santana Moss, Legedu Naanee, Malcolm Floyd, Mike Williams (TB), Dez Bryant Sit: Randy Moss, Michael Crabtree TE Start: Chris Cooley, Todd Heap, John Carlson, Jermichael Finley Sit: Jermaine Gresham, Owen Daniels, Tony Scheffler, Jeremy Shockey, Bo Scaife Kicker Start: La

Start/Sit Week 4 and a smattering of rankings

It's been awhile, my fantasy football friends. I have lofty intentions of getting several posts up here each week, but life is busy and I will do my best. So in the intervening 6 years since I've posted regularly on here, our family added another child (now there are 3!), moved back to Boulder for a sabbatical year and then returned to Knoxville, and we have basically been chugging along like many of the rest of you. My two boys (who turn 12 and 9 soon) are in a fantasy football league for the first time this year with their buddies, which ostensibly is the just the dads (and mom!) playing one another of course. But my oldest is taking it super seriously and what kind of Fantasy Football Librarian would I be if I didn't step up my game for him? So I'm back - and am interested in the fantasy landscape - lots of the same awesome sites out there, but many new ones as well. I'll try to incorporate a mix. Please be sure to let me know if my 2006 ways aren't working i