Skip to main content

Multi-year results for accuracy with QB rankings

Happy summer! To celebrate the new season and my return to a land where I can get e-mail and cell phone reception (had a great time this weekend but I do love the creature comforts of civilization...), I have a little something fun for everyone: the start of the aggregate accuracy analysis results. For those who are new to the blog, I've spent the past several years analyzing the accuracy of preseason expert rankings to find out which sites did the best job at predicting position rankings for the season. I started in 2007 and looked at 14 websites; after joining up with the Fantasy Sports Trade Association the following year it grew to 24 websites, and then a whopping 48 sites in 2009. At this point it has become almost entirely a submission process so if your favorite site isn't included, it's likely because they either chose not to submit or just do not know about the contest. You can read all of the details about how I analyze the rankings each year here and if you're a site that's interested in participating this year, stay tuned for more info on that front in August. Also, I want to note that one important condition of participation is that I keep all of the results of the contest confidential except for the top performers, so I unfortunately can't share the full results of each year's contest with you all. Besides I'd really prefer to focus on those who performed well year in and year out versus those who didn't do such a strong job with their rankings.

So I've had a plan to combine the results of the contest each year to create a multi-year assessment of each site, which is far more informative than just one year of results, and I finally had the time to do the analysis. Today we will take a look at quarterback rankings to see which site has been the most accurate with their preseason QB rankings from 2007-2009. Over the past 3 years I've included 54 different sites in my analysis, however only 10 sites have been included all 3 years, and and additional 13 sites have been tracked for 2 years, so for the multi-year analysis I'm focusing on sites that have participated in at least 2 years of the contest. Honorable mentions for those that did well in their 1 year of participation are noted below, too, though.

As you can see above, the contest has grown enormously and with the variation in the number of sites ranked each year, I decided to calculate a percentile ranking for each site each year. For example, Football Guys had the best QB ranking of the 14 sites I analyzed in 2007 so they were in the 7th percentile (1/14) - keep in mind this is not like the SATs; in this case you actually want a low percentile. After calculating the percentile for each year, I averaged each site's ranking from each year it was included in the FFLibrarian accuracy contest. The lower the number in the average column, the better the site did compared to their peers. Ask The Commish's 0.30 percentile means they averaged being in the 30th percentile in QB pre-season rankings over the last three years - that's the best of any of the sites I have data on for all three years. For comparison's sake, the site with the worst percentile ranking for QBs was averaging 0.86 for all three years (0.81 for sites with 2 years of data; 0.98 for those with 1 year).

Sites that have 3 years of participation
Ranking Site Average
1 Ask The Commish 0.30
2 Rotoworld 0.31
3 0.38
4 ESPN 0.44
5 FFToolbox 0.45

Sites that have 2 years of participation
Ranking Site Average
1 Football Guys 0.16
2 Fantasy Hulks 0.27
3 KFFL 0.34
4 0.43
5 Fantazzle 0.46

Sites that have 1 year of participation
Ranking Site Average
1 WhatIfSports 0.04
2 Junkyard Jake 0.06
3 0.08

I'm sure there are important things I'm forgetting to mention or notes that I should be making here, so please feel free to ask questions or make comments. As always, this is a work in progress so if you have a better method, please let me know but in the meantime a big kudos to my husband for talking all of this out with me. Just one of the many times it pays to have fallen in love with an economist/stats dude.

I'll be posting the results for RBs, WRs, and TEs over the next week or so in addition to my normal linking. Enjoy and a hearty congrats to Ask the Commish!


davinci78 said…
Well done and thanks for taking the time.
davinci78 said…
This comment has been removed by the author.
HughesNet said…
Good read. Interesting to see Rotoworld and outpacing the big boys across the positions. Interested in seeing more.

Popular posts from this blog

Back on the fantasy wagon

With my library conferences behind me (for a few weeks at least), I can now concentrate on fantasy football again. If you happen to be a librarian and are interested in how my co-presenter and I tied together fantasy football and information literacy, check out my co-presenter’s blog . Though I imagine most of you would much rather catch up on the following football news and thoughts for the week: I remain a bit dazed and confused by my certainty last summer that Cedric Benson was going to rock my RB2 world in 2007. Didn’t happen and now he’s moved onto ankle surgery, alleged intoxication and resisting arrest on boats. Awesome. You can practically taste Matt Forte’s value skyrocketing here. In fact, wondering about rookie RBs in general? Well then check out FF Xtreme’s The Day After: Ranking the Rookie Running Backs . I love reading articles or stats that challenge my preconceived notions – The Fifth Down’s Chad the Completion Percentage Machine does just that. No offense to thos

Streaming Defenses - Week 16

Don't know about you all, but in our house this week is just chock full of events, visiting with family, and getting organized for our upcoming time off over the holidays. Lots of scrambling going on around here and attempts to install some sense of gratitude in my boys. What am I grateful for this season? Many things but let's start with Bob. Bob has been my fantasy football rock, coming back week after week to post. Hats off to you, Bob, and thank you for your diligent work and research. So here he is with this year's final streaming defense post.  Well it’s the last week of the Fantasy Football Playoffs in most leagues, and my last post of the season. I hope you all found these posts useful. And if I failed you, you have to murder a lot more experts before you get to me. I myself have had a terrible playoff season. I’m out of all but one of my leagues due to the weird games the last two weeks. Between weather that didn’t materialize to normally consistent players co

Start/Sit Week 4 and a smattering of rankings

It's been awhile, my fantasy football friends. I have lofty intentions of getting several posts up here each week, but life is busy and I will do my best. So in the intervening 6 years since I've posted regularly on here, our family added another child (now there are 3!), moved back to Boulder for a sabbatical year and then returned to Knoxville, and we have basically been chugging along like many of the rest of you. My two boys (who turn 12 and 9 soon) are in a fantasy football league for the first time this year with their buddies, which ostensibly is the just the dads (and mom!) playing one another of course. But my oldest is taking it super seriously and what kind of Fantasy Football Librarian would I be if I didn't step up my game for him? So I'm back - and am interested in the fantasy landscape - lots of the same awesome sites out there, but many new ones as well. I'll try to incorporate a mix. Please be sure to let me know if my 2006 ways aren't working i