Skip to main content

Weekly accuracy assessment

I've got family in town right now so today's post is going to be brief - but awesome. So you all probably know that I do an annual accuracy assessment of experts' rankings, comparing how they've ranked players during to preseason to how those players performed at season's end. Many of you have asked if I'll be looking at the accuracy of weekly rankings, and unfortunately it hasn't been something I've been able to do due to the time involved...but now, my friends, there's a site that has filled that gap and is an incredible complement to the season-long accuracy assessment. Last week released their accuracy awards for weekly rankings and I for one am extremely excited about this. A big kudos to Andy Behrens from Yahoo for being the #1 overall most accurate expert with his weekly rankings from the 2009 season - for the full list of how the experts did, take a look at's Expert Accuracy Rankings - 2009.

Though just as important as the results is the methodology used - to see what they gathered and how they developed each expert's PAY (Prediction Accuracy Yield) and subsequent ranking, see the discussion here.

As a final cherry on top to this incredible project (and immense amount of work!), has also developed a Cheat Sheet Wizard, which is very similar to what I post each August with the average rankings position spreadsheet. I highly recommend you scope it out for updated consensus rankings...and drumroll includes PPR rankings for those who play in PPR leagues.

Basically, spend a bit of time on and you'll get lost in heavenly world of fantasy awesomeness. So have at it, let me know what you think!


dave said…
Hey Sara, I really appreciate the nice post and would love to hear any feedback from your members.

Now if I could just find some time to prep for my own draft, life would be really good. : )
Rogue33 said…
Very cool post. It's always nice to have more information that evaluates the experts. Makes it easier to find the sites to trust!
H00KemH02NS said…
Once again, I would throw an asterisk on all these rankings.

Ranked based on what scoring system? I'm not talking about PPR vs Standard, either. How much a passing TD nets has a whole lot to do with whether Tony Romo is more valuable than Jamaal Charles. How much are you punished for a fumble?

Just saying.
dave said…
Hey H00kem,

I think there are limitations to any study, but to ignore the results completely would suggest that there's zero insight to be had from analyzing this ranking data.

The scoring system and all the checks/balances/asterisks are available on our methodology page, which Sara was nice enough to link to.

At the very least, we're happy that the assessment has created some more discussion on the topic of "accuracy" in fantasy sports.
Randy said…
One comparison between the the weekly and the preseason, if I understand correctly... came in first for preseason rankings (Sara's analysis) but came in 19th (of 21) for weekly accuracy (FantasyPros' analysis).

What does that say? :-)

It's also worth noting that Yahoo! experts took 4 of the top 9 spots, overall. That's pretty significant.
dave said…
Hey Randy,

I won't speak for Sara, but from my perspective, I think you need to think of the two studies as complimentary to each other. And, I think it's important to note that the two studies use very different approaches to assessing accuracy.

I would say Sara's approach is more "pure accuracy" - did the expert hit or miss on the ranking of each player? While our assessment is more about trying to quantify the actual value that an expert's advice provides compared to his peers. There's no right or wrong, just two different ways to look at it.

On CBS specifically, we didn't run numbers for the draft rankings but I'm guessing they would have performed well, given how well they did with pure accuracy. There have been years when I personally have "killed the draft" and stunk it up during the season. Isn't it conceivable that CBS had the same kind of year?

Bottom line is, we need to continue to track everyone so we get a strong history of data. The more poker hands you play, the more likely you'll know whether you're really any good or not.

The Yahoo guys killed it. I'm happy for them because I think they get little respect...because they work for a big media company and not a "hard core" fantasy site.


Popular posts from this blog

Labor Day FF links

Happy long weekend! Here's hoping most of you are free to enjoy the 3-day weekend and the relaxation and fun that comes with it. I spent my Sunday participating in 3 fantasy football drafts scattered throughout the day and night, but I think I've finally wrapped up my drafts for the year. So like many of you I'm now turning my attention to the Week 1 games! But before we get to some player rankings and prep for Week 1, I want to make sure that all you experts out there are aware of the upcoming deadline for the annual accuracy contest that I run with the Fantasy Sports Trade Association. I compare preseason positional rankings from experts to the final outcome of the fantasy season to see which site had the most accurate preseason rankings, and this year's deadline is fast approaching: September 9th by kickoff. Check out the info on the FSTA site for more details and be sure to e-mail me your submissions. Now, onto Week 1! First up, let's get to some of this

2010 Week 2 consensus start/sit

Greetings and happy weekend. I've got this week's consensus start/sit for you which is a tally of all the start/sit votes that players received in the various start/sit articles that I linked to earlier in the week. So while these aren't my personal recommendations, I do think knowing what the consensus is could be of help when making lineup enjoy! Note: the players are listed in no particular order. QB Start: Michael Vick, Kyle Orton, Matt Ryan, Jason Campbell Sit: David Garrard, Alex Smith, Jay Cutler, Vince Young, Carson Palmer RB Start: LeSean McCoy, Darren McFadden, Ahmad Bradshaw, Brandon Jackson Sit: Tim Hightower, C.J. Spiller, Beanie Wells, Ronnie Brown WR Start: Santana Moss, Legedu Naanee, Malcolm Floyd, Mike Williams (TB), Dez Bryant Sit: Randy Moss, Michael Crabtree TE Start: Chris Cooley, Todd Heap, John Carlson, Jermichael Finley Sit: Jermaine Gresham, Owen Daniels, Tony Scheffler, Jeremy Shockey, Bo Scaife Kicker Start: La

Start/Sit Week 4 and a smattering of rankings

It's been awhile, my fantasy football friends. I have lofty intentions of getting several posts up here each week, but life is busy and I will do my best. So in the intervening 6 years since I've posted regularly on here, our family added another child (now there are 3!), moved back to Boulder for a sabbatical year and then returned to Knoxville, and we have basically been chugging along like many of the rest of you. My two boys (who turn 12 and 9 soon) are in a fantasy football league for the first time this year with their buddies, which ostensibly is the just the dads (and mom!) playing one another of course. But my oldest is taking it super seriously and what kind of Fantasy Football Librarian would I be if I didn't step up my game for him? So I'm back - and am interested in the fantasy landscape - lots of the same awesome sites out there, but many new ones as well. I'll try to incorporate a mix. Please be sure to let me know if my 2006 ways aren't working i