Skip to main content

Weekly accuracy assessment

I've got family in town right now so today's post is going to be brief - but awesome. So you all probably know that I do an annual accuracy assessment of experts' rankings, comparing how they've ranked players during to preseason to how those players performed at season's end. Many of you have asked if I'll be looking at the accuracy of weekly rankings, and unfortunately it hasn't been something I've been able to do due to the time involved...but now, my friends, there's a site that has filled that gap and is an incredible complement to the season-long accuracy assessment. Last week released their accuracy awards for weekly rankings and I for one am extremely excited about this. A big kudos to Andy Behrens from Yahoo for being the #1 overall most accurate expert with his weekly rankings from the 2009 season - for the full list of how the experts did, take a look at's Expert Accuracy Rankings - 2009.

Though just as important as the results is the methodology used - to see what they gathered and how they developed each expert's PAY (Prediction Accuracy Yield) and subsequent ranking, see the discussion here.

As a final cherry on top to this incredible project (and immense amount of work!), has also developed a Cheat Sheet Wizard, which is very similar to what I post each August with the average rankings position spreadsheet. I highly recommend you scope it out for updated consensus rankings...and drumroll includes PPR rankings for those who play in PPR leagues.

Basically, spend a bit of time on and you'll get lost in heavenly world of fantasy awesomeness. So have at it, let me know what you think!


dave said…
Hey Sara, I really appreciate the nice post and would love to hear any feedback from your members.

Now if I could just find some time to prep for my own draft, life would be really good. : )
Rogue33 said…
Very cool post. It's always nice to have more information that evaluates the experts. Makes it easier to find the sites to trust!
H00KemH02NS said…
Once again, I would throw an asterisk on all these rankings.

Ranked based on what scoring system? I'm not talking about PPR vs Standard, either. How much a passing TD nets has a whole lot to do with whether Tony Romo is more valuable than Jamaal Charles. How much are you punished for a fumble?

Just saying.
dave said…
Hey H00kem,

I think there are limitations to any study, but to ignore the results completely would suggest that there's zero insight to be had from analyzing this ranking data.

The scoring system and all the checks/balances/asterisks are available on our methodology page, which Sara was nice enough to link to.

At the very least, we're happy that the assessment has created some more discussion on the topic of "accuracy" in fantasy sports.
Randy said…
One comparison between the the weekly and the preseason, if I understand correctly... came in first for preseason rankings (Sara's analysis) but came in 19th (of 21) for weekly accuracy (FantasyPros' analysis).

What does that say? :-)

It's also worth noting that Yahoo! experts took 4 of the top 9 spots, overall. That's pretty significant.
dave said…
Hey Randy,

I won't speak for Sara, but from my perspective, I think you need to think of the two studies as complimentary to each other. And, I think it's important to note that the two studies use very different approaches to assessing accuracy.

I would say Sara's approach is more "pure accuracy" - did the expert hit or miss on the ranking of each player? While our assessment is more about trying to quantify the actual value that an expert's advice provides compared to his peers. There's no right or wrong, just two different ways to look at it.

On CBS specifically, we didn't run numbers for the draft rankings but I'm guessing they would have performed well, given how well they did with pure accuracy. There have been years when I personally have "killed the draft" and stunk it up during the season. Isn't it conceivable that CBS had the same kind of year?

Bottom line is, we need to continue to track everyone so we get a strong history of data. The more poker hands you play, the more likely you'll know whether you're really any good or not.

The Yahoo guys killed it. I'm happy for them because I think they get little respect...because they work for a big media company and not a "hard core" fantasy site.


Popular posts from this blog

Back on the fantasy wagon

With my library conferences behind me (for a few weeks at least), I can now concentrate on fantasy football again. If you happen to be a librarian and are interested in how my co-presenter and I tied together fantasy football and information literacy, check out my co-presenter’s blog . Though I imagine most of you would much rather catch up on the following football news and thoughts for the week: I remain a bit dazed and confused by my certainty last summer that Cedric Benson was going to rock my RB2 world in 2007. Didn’t happen and now he’s moved onto ankle surgery, alleged intoxication and resisting arrest on boats. Awesome. You can practically taste Matt Forte’s value skyrocketing here. In fact, wondering about rookie RBs in general? Well then check out FF Xtreme’s The Day After: Ranking the Rookie Running Backs . I love reading articles or stats that challenge my preconceived notions – The Fifth Down’s Chad the Completion Percentage Machine does just that. No offense to thos

Streaming Defenses - Week 16

Don't know about you all, but in our house this week is just chock full of events, visiting with family, and getting organized for our upcoming time off over the holidays. Lots of scrambling going on around here and attempts to install some sense of gratitude in my boys. What am I grateful for this season? Many things but let's start with Bob. Bob has been my fantasy football rock, coming back week after week to post. Hats off to you, Bob, and thank you for your diligent work and research. So here he is with this year's final streaming defense post.  Well it’s the last week of the Fantasy Football Playoffs in most leagues, and my last post of the season. I hope you all found these posts useful. And if I failed you, you have to murder a lot more experts before you get to me. I myself have had a terrible playoff season. I’m out of all but one of my leagues due to the weird games the last two weeks. Between weather that didn’t materialize to normally consistent players co

Start/Sit Week 4 and a smattering of rankings

It's been awhile, my fantasy football friends. I have lofty intentions of getting several posts up here each week, but life is busy and I will do my best. So in the intervening 6 years since I've posted regularly on here, our family added another child (now there are 3!), moved back to Boulder for a sabbatical year and then returned to Knoxville, and we have basically been chugging along like many of the rest of you. My two boys (who turn 12 and 9 soon) are in a fantasy football league for the first time this year with their buddies, which ostensibly is the just the dads (and mom!) playing one another of course. But my oldest is taking it super seriously and what kind of Fantasy Football Librarian would I be if I didn't step up my game for him? So I'm back - and am interested in the fantasy landscape - lots of the same awesome sites out there, but many new ones as well. I'll try to incorporate a mix. Please be sure to let me know if my 2006 ways aren't working i