2009 Forecast Evaluations

4 Comments
February 9th, 2010 by Mays
Categories: Other Sites, Projections

Jared Cross has recently taken the time to see how well last year’s projections matched up with what really happened in 2009.

Here are the correlations they observed in fantasy (SGP) terms:

  1. Averaged Projections
  2. Fantistics
  3. The Sporting News
  4. Marcel
  5. Steamer
  6. ZiPS
  7. CHONE
  8. PECOTA
  9. 2008 Stats

Averaging all the projections together produced the best results, but the next best options were the Fantisticsand TSN projections.

For projection rate stats, it was the completely computerized CHONE and ZiPS projections that did the best job. However, for fantasy, it is even more important to have accurate playing-time projections than to just forecast a player’s ability. So the systems that had human-created playing-time forecasts (Marcel and Steamer used community playing-time projections) ended up on top in fantasy rankings.

Unsurprisingly, predicting 2009 stats based on 2008 ended up at the bottom, but PECOTA did pretty terribly, as well. Rumor is that Baseball Prospectus has done some work to get it back in shape for 2010.

The Fantistics projections are new to me, but that’s because they cost money ($40). It could be something worth looking at for people who are willing to pay for a fantasy resource, though.

It looks like our idea of “composite” projections is on the right track for accurate fantasy values. And combining the free, computer-generated projections with human-developed, playing-time forecasts looks to be ideal.

Related posts:

4 Responses to “2009 Forecast Evaluations”

  1. hankp101 says:
    February 9, 2010 at 11:29 am

    Mays
    Great work on the site as always. Do you anticipate adding any additional projection systems to the Price Guide this year?

  2. Tyler says:
    February 9, 2010 at 1:56 pm

    Any thoughts of combining the averaged CHONE/ZiPS rate stats with FanGraphs’ fan playing time projections?

  3. Mays says:
    February 9, 2010 at 6:18 pm

    @hankp101: Dan has allowed me to use ZiPS in the past, and so I’d like to add it when it is ready. I don’t know of any others that are free to use.

    @Tyler: That is exactly what I meant with my closing statement above about what would be “ideal.” :-)

  4. Nick J says:
    February 10, 2010 at 12:14 am

    Very cool. Thanks for posting this. I wonder where CAIRO fits in with the above. Guess we’ll see next year.

Leave a Reply