Picking what projection system you want to use in fantasy baseball isn’t like picking out a beer. With beer, there’s really not a wrong choice; there’s just preferences. With projections, you can certainly make a wrong choice by choosing a less accurate system and, thus, having a crappy baseball season.
Last week, we looked at the 2013 hitter projections and learned that Steamer hasn’t lost their magic and continues to set the bar for hitter projections. We also saw that my weighted averages worked out very well too for fantasy baseball purposes.
We turn our attention to the pitchers now. Last year, it was Steamer and the Fangraphs Fans that took the top spots. 2013 was also the first year for a new combination between Steamer and the Fangraphs Fans (Steamer+Fans). For the past year I’ve been curious to see if this amalgam of the two projections would create a pitching projection powerhouse. Let’s find out.
The Method
As noted in the hitter analysis methodology, I standardize all of the projections for each statistic so that I look at the predicted z-score in that stat for each player (z-score being how many standard deviations above/below the mean that projection was). This helps to determine how high above or below average a projection was in that system for a player versus reality.
For the pitcher analysis, I also chose to use Mean Absolute Error (MAE) to compare results. Another popular choice for this type of analysis is Root Mean Squared Error (RMSE) but MAE won’t penalize large mistakes as harshly as RMSE which makes sense for fantasy baseball since if a projection is way off then you can bench a player. Mean Absolute Error is simply the average difference between the projected z-score and the true z-score from the actual 2013 results within each stat.
I only included players in this analysis that were showing up in drafts last pre-season and were shared among all of these analyzed projection systems. This is ultimately an analysis of how valuable the projections were on draft day last year but I did remove players who ended up suffering major injuries that made their season basically non-existent. This left 189 pitchers in my pool of players from last season.
The 2013 Rankings
For a full rundown of the competitors, see my post that introduces the baseball projections.
After performing the analysis as stated above, here are the rankings for how well each projection system performed:
| W | K | ERA | WHIP | WERTH* | |
| CAIRO | 1 | 5 | 10 | 10 | 10 |
| Steamer | 5 | 7 | 2 | 2 | 2 |
| Fangraphs | 2 | 1 | 4 | 4 | 3 |
| ZiPS | 7 | 3 | 5 | 9 | 4 |
| Oliver | 9 | 8 | 8 | 8 | 8 |
| Marcel | 8 | 9 | 9 | 5 | 9 |
| MORPS | 6 | 6 | 7 | 6 | 7 |
| Clay | 10 | 10 | 6 | 7 | 6 |
| Steamer-Fans | 4 | 2 | 1 | 1 | 1 |
| Special Blend | 3 | 4 | 3 | 3 | 5 |
*WERTH is the value of adding the four roto z-score values together for a player to get their total projected rotisserie value. It should be noted that I did not include Saves in this projection analysis despite it being a roto category. To me, it’s about player opportunity and coaches’ whimsy that is beyond the power of projection.
Though rankings tell us how well each system did, it doesn’t illustrate the gaps between the places so here’s a visual representation of the rankings that shows precisely how above or below average each system was in comparison to one another.
Trends or Anomalies
Despite a strong performance with the hitters, my “Special Blend” actually whiffed here and couldn’t keep pace with the Steamer projections, Fangraphs projections nor those mighty Steamer+Fans projections. Some tweaks may need to be done for future years as I would expect a weighted average of the best projection systems to perform better than this.
The Fans seemingly did well with the counting stats tied into playing time such as Wins and K’s while Steamer did well with the rate stats like ERA and WHIP. Combining those two powers resulted in a very accurate Steamer+Fans projection system in 2013. The Fans helped boost the prediction power of Steamer by providing more accurate playing time projections. For hitters, the combination didn’t work quite as well but it may be that it is easier for humans to predict pitcher playing time.
One oddity to note is that CAIRO dominated in projecting Wins for 2013 but was the worst in the other rate-based projections. I’m not sure exactly what happened there that made them so strong in one area but so weak in others but it’s an interesting anomaly to make note of. I don’t think this is a trend that would continue though as CAIRO actually finished dead last in Win projections the year prior so this may just be a weird fluke.
The Awards
I wondered aloud last year in my projection analysis about how a combination between Steamer and Fangraphs would do. Now we know. It performs awesomely, for lack of a better word. The non-combined versions of Steamer and Fangraphs themselves still take the other top honors but the combination of the two resulted in the best projection system for pitchers in 2013.
Moving forward, I will be tweaking my Special Blend formula a little bit to hopefully get into the top spots for 2014. But, for the year ahead, you might be in great shape if you rely on the Steamer+Fans projections for your pitchers. Fans will take care of the playing time intricacies while Steamer will bring the great rate stat projections and you will reap the benefits of both.
