VORP, ERA and FIP projections (keeping it simple)

In honor of Bill Simmons (one of my favorite sportswriters) getting on board with sabermetrics, I'm gonna keep this one simple. As Bill put it, VORP is a great stat because "it's not easy to figure out, but it's easy understand!" So I'll try to keep the intro quick, and maybe even interesting. If you prefer, just skip forward to the 2010 projection, or you can read a (abridged) explanation of what I did.

Estimating ERA via FIP and VORP via ERA

For my IP projections, I looked at many factors in the player's past performance. Then I looked at his injury data to tweak those IP projections. Doing so, I got a somewhat complicated system that performs considerably better than PECOTA or CHONE.

However to estimate value, I took a simpler approach. I projected FIP (Fielding Independent Pitching [ERA approximator]), then used that value to calculate ERA.

Given a projection for ERA, I take my injury-based IP projection, and get a projection for VORP (Baseball Prospectus's runs above replacement level stat). I do not make any adjustments for league, park factors, quality of opposition, or whether innings where pitched in start or relief.

Now, to make sure all of this works, I compare my ERA and VORP projections to PECOTA's and CHONE's preseason projections for 2009. I compute VORP (with no park or league adjustments) from PECOTA's and CHONE's projections for IP and ERA, thus making them comparable to my VORP projections.

Comparisons to PECOTA and CHONE

The ERA comparison results are simple:

• My 2009 ERA projections (all weighted by actual IP) are better than PECOTA and equal (50%-50%) to CHONE.
• CHONE is better that my system (56%-44% with smaller average error) if we only look at 60IP+ pitchers.

Therefore, my simple ERA projection system is better than PECOTA, and slightly better than CHONE for fringe guys, but slightly worse for true major leaguers (who managed not to get hurt too much). And this is with my system treating all players equally, without park or league adjustment, and without using minor league stats of any kind.

When I combine my superior IP projection system with an ERA-projection system that's a little worse than CHONE, I have a superior system for projecting VORP (or any other runs over replacement stat). Whether we look at all projected pitchers, those pitchers projected for 60IP+ (by either system), or those projected for 120IP+, my VORP approximations are better then CHONE or PECOTA by a significant margin.

I also do significantly better (60%-40% with 10% lower average error) compared to PECOTA when using the final VORP figures that BP itself published (rather than the ones that I approximate with a static 5.8 replacement-level RA and static 0.93 RA/ERA ratio).

I'll probably have the details for all these tests up in the future, but just take my word for it for now.

The Point

I'm not arguing that my ERA approximations are amazing. They don't look at anything other than the last three years' FIP, QERA (BP's fielding independent stat that ignores HR%), start/relief ratio and rookie status. My ERA approximations ignore league and park effects on ERA entirely. Rather, I admit that CHONE probably has better ERA approximations that I do at this point.

However, if you want to know what value a player will have to his team (in runs saved, or otherwise), it's important to have a good projection of playing time. CHONE and PECOTA over-estimate IP across the board, but they do so especially for young players. The chance of injury or collapse in performance is always there, and needs to be a part of the projection process.

For every Mariano Rivera and CC Sabathia (consistently good), there is an occasional Brandon Webb (unexpected injury) or Brad Lidge (collapsed value) or Cole Hamels (very bad luck with BABIP). These cases seem rare when they happen, but we have cases like these every season. Therefore, a projection system has to either catch these cases more often than not, or it has to temper high expectations across the board. My IP system does a little of both. Even combined with simple ERA projections, I've still got a very good projector of overall pitcher value.

2010 projections

Follow this link to see IP, ERA, FIP and VORP projections for all pitchers who either pitched in 2009, or threw 120IP in 2008 and don't seem to be officially retired yet (ie Ben Sheets).

For those not familiar with some of the stats above, here is a quick reference below. I think they are pretty intuitive, even if you don't know how they are computed. As Bill Simmons wrote, "I don' care how you killed the cow; just give me a great steak!" These projections is what's for dinner, in time for the season opener.

Stats I'm projecting

FIP (Fielding Independent Pitching):

• Approximates ERA by using only the strikeout rate (SO9), walk rate (BB9) and home run rate (HR9)
• FIP = (13*HR9 + 3*BB9 - 2*SO9)/9 + 3.2

VORP (Runs (saved) Over Replacement):

• How many runs was the pitcher worth, compared with those same innings going to a replacement-level (AAA) player?
• Replacement level varies by league and is different for starter and relievers.
• I just set it at ERA_replacement = 5.4; RA_replacement = 5.8 (post 2005 data) when I'm lazy.
• Since 10 runs ≈ 1 win, a 35.0 VORP player is worth about 35.0/10 = 3.5 wins to his team. A replacement-level player is worth nothing, independent of playing time.

Using the fact that VORP translates directly into runs and wins, it's a nice stat for approximating how well a team's pitching staff will do as a whole. Since we assume that guys not on the season-opening 25-man roster or the DL are probably "replacement level," we can approximate the value of the rotation as a whole. We add the VORP projections for the top guys, and don't worry about who the excess innings will go to.

Since my IP projections take account of injury histories, and VORP is proportional IP, my VORP projections take injury histories on the staff into account. Individual projection can often look strange, but since those projections average out, staff projections should be more accurate that projections for individual pitchers.

I intend to have the staff projections for a couple of teams tomorrow.

Odds & Ends

There is a better stat than VORP for approximating wins added called WAR (Wins Above Replacement). It's similar to VORP, but takes into account a few important factors that VORP does not. However I didn't have time to generate WAR projections. I'll try to get to that later today or tomorrow before the season starts.