Starter vs Reliever

I was looking at ways to show (in a chart or map) how pitcher performance changes when pitcher change their type (ie learn or forget different pitches). I soon realized that I needed a single rate stat to measure how a group of pitchers' performance changes. Of course, this stat has to be FIP. However, there are starter/reliever issues that need to be considered first.

This problem has been considered before, and I not aim to shed much new light on it, except to show a couple of graphs based on recent data.

To establish the difference between starter and reliever performance, there is a long, comprehensive article from Steve Treder on THT that often gets cited. It looks at starter/reliever differences throughout baseball history, and concludes that the difference between starter and reliever performance is consistently about 8% of ERA (about 0.3-0.4 on the ERA scale). However that is not the question I need to answer. Instead, I want to compare a group of pitchers' collective performances, and adjust for the groups' tendencies to be throw innings as starters or as relievers. Since starters are, collectively, better pitchers than relievers, the adjustment has to be larger than 8%.

A more useful study is Tom Tango's work on his blog. He looks at the same pitchers as starters and relievers, making several important adjustments. His conclusion is that pitchers have 17% more strikeouts, and 17% fewer home runs as relievers, as they do as starters. They also have a 17% better BABIP. He concludes that the same pitchers are "about a 1 run per 9IP" RA better as relievers. I did something simpler (although less thorough), and came up with much of the same conclusions.

Consider the graph below. I mapped IP to FIP by bucketing real pitcher seasons by IP (2005-2009 data). The graph plots the median FIP for each bucket, along with the median start percentage (% of innings thrown by individual pitchers in starting roles) for each bucket.

By looking at the median, rather than the average, it is easy to see where the transition from starters to relievers really takes place (as measured by IP). This is like chemistry class. As we move up the IP scale, the average pitcher has higher energy. However he needs to overcome a state change to move from full time reliever to full time starter. The 60-120 IP buckets find him in a state of transition. Supposing that these pitchers are all of roughly the same ability, and their FIP is different only due to role changes, I use the data to fit the following function (where start % is on a 0-1 scale):

trFIP = a + (start %) * b

Here "trFIP" represents a "translated FIP," which I assume to be constant, on average, throughout the transition. The best fit is for a = 3.92 and b = 0.81. Therefore for any pitcher:

trFIP = FIP + 0.81 (1 - start %)

Now FIP is meant to reflect the pitcher's skills at striking out hitters and at (not) giving up walks, but it also predicts ERA very well (at least for FIPs near the league average). The translated trFIP does not have this property. As Tom Tango showed, BABIP changes along with SO9 and HR9, so ERA increases more than FIP increases as relievers become starters. A similar exercise for ERA led me to this formula:

trERA = ERA + 1.19 (1 - start %)

However, trERA is not very useful in comparing talent levels between groups of pitchers. I can explain why, but let's get back to comparing groups of pitchers...

Back to trFIP, here is the same graph as before, but also with trFIP buckets by IP:

The graph between 60IP and 120IP is not quite flat. This is not surprising, as I am taking the median of individual trFIP values within the buckets. Some pitchers get the full 0.81 penalty, but not all do. I am taking the median of a combined distribution.

My study is not as thorough as Tom Tango's, but I like my results. My median trFIP buckets imply that your typical 60-80 IP pitcher from 2005-2009 is a little bit better than a typical 100-120 IP pitcher from the same time period. I think this is actually true. A 60-80 IP pitcher is very likely to be a high-IP reliever, such as a closer or setup man. He could also be a swing man/long man, but bear with me. A 100-120 IP pitcher is likely a back-end starter or also a swing man/long man who ended up filling in for an injured regular. Although the former is likely a failed starter, he must have found success in the bullpen to get such heavy use. At the very least, it is plausible to suggest that the top-end relievers are better overall pitchers than low-end starters, if only slightly.

The again, I might be reading too much into a small matter. Going forward, I will use trFIP as a simple measure of overall performance, which is not biased by changes in starter/reliever usage. It creates a useful benchmark, but is not meat to as an absolute definition of differences between starters and relievers.

Here is the same graph of average performance by pitcher type, but using trFIP instead of FIP:

The type 1 (cutter throwers) pitchers are still collectively the best of the groups, but the difference between type 2 (slider throwers with secondary pitches) and type 6 (slider throwers without secondary pitches) is eliminated by the differences in the trFIP adjustment.

Also trFIP shows type 0 pitchers and type 3 pitchers to be largely ineffective. Type 0 pitchers are those whose core pitch is a change-up, but who also throw other pitches, often sliders (as opposed to type 5 pitchers who throw change-ups, but with curveballs as secondary offerings). Type 3 pitchers throw both a slider and a curve, or possibly a slurve. 

Type 3 pitchers are similar to both the type 2 slider-throwers and the type 4 curveball-throwers, but are more likely to become type 2 pitchers. Average trFIP seems to suggest that moving from type 3 to type 2 is not a bad move. This may just be confirmation bias, but the trFIP graph seems to suggest that throwing a slurve is, indeed, not an effective way to pitch. The fact that only 4% of pitchers feature both a slider and a curveball (but 70% featured one or the other) might also suggest the same. To be fair, successful type 3 pitchers Chris Carpenter and Adam Wainwright throw both a hard slider, and a slow curve. They are not throwing slurves. But most pitchers find more success with one pitch or the other, it seems.

Now that I have a way of comparing pitchers without concern for starter/reliever issues, I will look into what happens when pitchers switch types. Expect more charts, and maybe a confusion matrix.

Performance by Type (Intro)

Here is a graph of average pitcher performance by pitcher type, weighed by IP (over all pitchers with 20IP+ seasons).

There are non-trivial differences in strikeout rates, walk rates, and resulting FIP (HR9 rates are pretty constant, with a slight decrease for high-strikeout pitchers). However, there is not enough information in this one chart to jump to conclusions about what pitches to teach your 15-year old future major leaguer. Most clearly, pitchers can be successful by employing the methods of any of the eight pitcher types that I identified.

With selection bias playing a role, it is hard to conclude much from the raw differences between the performance of the pitcher types. However, the strikeout rates for type 6 and type 7 pitchers stand out, as compared to type 2 pitchers. Type 6 and type 2 are both fastball/slider pitchers, and yet type 6 pitchers have significantly higher strikeout rates. Type 6 pitchers surrender more walks, but still have lower FIPs than type 2 pitchers. However this difference can be explained by the facts that: type 6 pitchers throw harder fastballs on average (91.1 mph vs 90.1 mph), and they are much more likely to throw in relief (32% of innings vs 89% of innings).

Therefore, while it would be wrong to claim that type 6 pitchers are more effective then type 2 pitchers, it is fair to suggest that hard throwing fastball/slider pitchers (ie type 6) can be effective major leaguers by recording high strikeout rates, even if they can not control their walks very well. However, they are more likely to find success in relief roles with that repertoire.

I've often read that pitchers who are wild, inconsistent, or have limited repertoires should be relievers. I've also read that pitchers have the most value as starters, regardless of what they throw, as long as they guys out. Joba Chamberlain has become somewhat of a contentious issue in this regard. I have read some respected writers claim emphatically that he should be a reliever (because of his stuff, and his temperament), and others just as strongly claim that he has the most value as a starter (even if his performance per inning suffer from the move).

In earlier posts, I wrote that Chamberlain's dramatic drop in strikeout rates (from 10.6 SO9 in 2008 to 7.6 SO9 in 2009) can be explained largely by his drop in average fastball speed (from 95.0 mph to 92.5 mph). I argued that he never over-performed his expected SO9, relative to his fastball speed. Pitchers who throw 95 mph on average are supposed to have really high strikeout rates!

Now Joba Chamberlain is a typical type 6 pitcher, thinking about becoming a type 2 pitcher. He was a strong type 6 in 2008 (as well as in his magical 2007 debut). He became strongly type 2 in 2009 by throwing more curves, and adding a change-up. I guess he needed to expand his repertoire to make it as a starter.

In 2010, he is back to being a typical type 6 pitcher. Joba is throwing 65% fastballs, 32% sliders, 3% curves, and no change-ups. As a full time reliever, he has been pretty good so far (3.28 FIP and 8.7 SO9, albeit in only 10.1 innings). His fastball velocity isn't what it used to be, but it's up at tick at 93.1.  The jury's still out on Chamberlain, but if I had to guess, I think he's a type 6 pitcher to stay. He can still be effective as reliever without a 95 mph average fastball, but he may unfortunately have peaked.  That is strange to suggest for a guy who is only 24 years old, but he has lost some valuable velocity, and has not added much to his skill set.

Going forward, I will use my classifications to look at other pitchers' careers. This should illuminate some trends that simple averages of different pitcher types can not tell us.

Thankfully, pitchers change types quite often. In the last post, I mentioned some pitchers who have kept a consistent type for a long time. However, some pitchers re-invent themselves several times over a career. Rich Harden started out as as a type 2 (slider with secondary), then was a type 7 (splitter with slider) for several years. More recently, he is a type 5 (change-up with curve) pitcher. All Star closer Ryan Franklin has been a type 6 (fastball/slider only), type 2 (slider with other pitches), type 7 (splitter) and type 1 (cutter) in full seasons since 2002. This year, he is throwing lots of curve balls, and might end up as a type 4 (curveball with change-up backup)!

I look forward to analyzing this data more systematically. If you are interested in pitcher classifications for 2002-2009 (20IP+ seasons), they are alphabetically listed here. I have a notion of "weak membership" and "strong membership" for the categories, but I have not finalized it yet, so it's not included.

Pitchers: Eight Different Kinds

I often read that: "pitcher X is similar to pitcher Y, so he should a) have similar performance b) age at the same rate c) experience a similar injury history." Sometimes the comparisons are based on height, weight, handedness, or otherwise, but usually we think of pitchers as being similar, if they deliver similar performance. Without advanced stats, we might consider pitchers similar if they had similar won-loss records or had similar strikeout rates. Now, some of us might think of pitchers as being similar if they are fly ball pitchers or ground ball pitchers, or if their fastball moves inside or outside to right-handed hitters. However, these categories all seem to reflect performance, to varying degrees.

My goal is to classify pitchers, in a performance-neutral way, based on the pitches that they throw. I will present eight categories that comprehensively classify pitchers into types, based almost exclusively on the percentages that they throw their pitches (collected for 2002-2009 by BIS and for 2007-20010 by PitchF/X, all viewable on FanGraphs).

Why Classify by Pitches?

This is not to say that classification by factors other than pitch selection isn't interesting. We already classify pitchers by role (starter/closer/setup man) and by performance, in casual baseball conversations. However, I think it would be useful to classify pitchers by how they pitch, rather than by their results. In order to ask questions like "how do curveball pitchers age relative to other pitchers?" and "do fastball/slider pitchers make good starters?" it is necessary to have a way to classify pitchers based on what they throw, in a comprehensive and non-arbitrary way. (Declaring that a pitcher is a curveball pitcher if and only if he throws 25% curveballs seems too arbitrary, and unnecessarily restrictive.)

Why Eight Categories?

Baseball Info Solutions (BIS) tracked & classified a large number of pitches for 2002-2009, giving us insight into how each pitcher's repertoire broke down over those years. This is the data that I use to derive my classifications. Since 2007, MLB has contracted Pitch F/X (same guys who do the virtual first-down line for NFL games) to track pitches in all major league stadiums. Their data is more comprehensive (Pitch F/X also tracks two-seam (sinking) fastballs), but it is available on a much smaller sample. (All of this data is hosted on FanGraphs and can be viewed for free.)

BIS recognizes seven pitches: fastball, slider, cutter, curve, change-up, splitter and knuckleball. A few months ago, I developed a system to determine which of these were a pitcher's "core pitches," which were "secondary pitches," and which were pitches that he did not really throw (even if they had pitch counts over 0% according to BIS. Consider a pitcher who throws 70% fastballs, 25% sliders, and 3% change-ups and 2% curveballs. His core pitches are fastball and slider, while the others are not relevant. Now if he threw 70% fastballs, 20% sliders, 9% change-ups and 1% curveballs, we might says that fastball and slider are still his "core pitches," but "change-up" is now a "secondary offering." I classify pitchers based on these distinctions.

Over the course of a season, almost all pitchers' repertoires can be described as:

• core offerings: Fastball + 0-2 other pitches
• secondary offerings: 0-3 other pitches 

Actually, most pitcher seasons fall into an even tighter rule:

• core offerings: Fastball + 1 other pitch
• secondary offerings: 0-2 other pitches

Therefore, the simplest way to describe a pitcher's repertoire is to name his core offering (ie the pitch he throws most often, not including his fastball). There is a strong correlation between the core offering and his likely secondary pitches. Pitchers who throw lots of curveballs tend to throw some change-ups, as well. Pitchers who throw splitters usually also throw sliders.

Since there are six possible pitches available as core offerings (not including the fastball, which everyone throws), there are six obvious categories. However, the knuckleball is so rare in today's game that I do not create a category based on it. Instead, there are five obvious categories (slider, cutter, curve, change-up, and splitter). In practice, more categories are needed, since not all pitchers who throw fastballs and sliders are truely similar, etc.

Using a simple clustering algorithm (SimpleKMeans using manhattan distance for the "core" and "secondary" pitches), I came up with seven categories. This was enough to get a category focused on cutters. I still needed a category for splitters, so I created one manually (by computing a centroid for all pitchers who throw a splitter). Then I tuned the weights and added other small considerations, so that the selection criteria for the categories were more intuitive.

Eight Pitchers

Here are the eight pitcher categories that I came up with, along with explanations, examples, and obvious tendencies.

Type 0: Changeup (Slider & Curve backup)

(14% of pitcher seasons)

The types start with 0, since nerds number their days 0-6.

A typical type 0 pitcher throws 61% fastballs, 18% change-ups, 11% sliders, and 6% curves. They are most similar to type 5 pitchers, although they are more likely to throw the slider, less likely to throw the curve, and less likely to be starting pitchers.

Sucessful type 0 pitchers include:

• Pedro Martinez (until 2006, when he became a type 5)
• Felix Hernandez
• Johan Santana (until 2007, when he became a type 5)

Type 1: Cutter (no common backup)

(4% of pitcher seasons)

A typical type 1 pitcher throws 56% fastballs, 20% cutters, and various other pitches depending on the pitcher. Type 1 pitchers are not similar to any other pitchers, but they are more likely to throw curveballs that sliders, which makes them somewhat similar to type 4 and type 5 pitchers. Type 1 pitchers are likely to be starters, but some very successful closers are type 1 pitchers.

Successful type 1 pitchers include:

• Mariano Rivera
• Jon Lester
• Andrew Bailey
• Roy Halladay since 2007 (previously a type 4 pitcher)

Type 2: Slider (Change-up and Curve backup)

(18% of pitcher seasons)

Type 2 pitchers are fastball/slider pitchers, who also throw other pitches at non-negligible rates. A typical type 2 throws 62% fastballs, 18% sliders, 10% change-ups and 7% curves. Type 2 pitchers are similar to type 6 pitchers, but type 2 pitchers are those that also have secondary offerings. Type 2 pitchers are more likely to be starters than type 6 pitchers, and also don't throw as hard.

Successful type 2 pitchers include:

• CC Sabathia
• Zack Greinke
• Jake Peavy

Type 3: Curve & Slider (no common backups)

(4% of pitcher seasons)

Type 3 pitchers throw both a curve and a slider frequently enough for them to both be considered "core pitches." A typical type 3 pitcher throws 57% fastballs, 18% curves and 16% sliders.

When I first saw this category, I thought these might be pitchers who throw a "slurve," or those pitchers who switched breaking pitches within a season. Those would end up in this category, to be sure, but successful type 3 pitchers recently included some very good pitchers:

• Chris Carpenter (2005 and 2009)
• Adam Wainwright (2005 and 2009, type 2 in between)
• Matt Morris

Type 4: Curve (with Change-up backup)

(16% of pitcher seasons)

A Type 4 pitcher throws 64% fastballs, 21% curves, and 9% change-ups. These pitchers typically throw very few sliders. They are most similar to type 5 pitchers (who throw more change-ups), although a type 4 pitcher can become a type 3 pitcher if he starts to throw more sliders. 

Successful type 4 pitchers include:

• Roy Halladay before 2006 (now he's a type 1 pitcher)
• Roy Oswalt
• Ben Sheets
• Justin Verlander 

Type 5: Change-up (Curve backup)

(8% of pitcher seasons)

A typical type 5 pitcher throws 62% fastballs, 23% change-ups, and 12% curves. Type 5 pitchers are similar to type 0 pitchers in that both rely heavily on the change-up. However type 5 pitchers throw more curveballs, while type 0 pitchers throw silders. Also type 5 pitchers are the softest-tossing category that I considered, with a typical average fastball of only 89.1 mph.

Successful type 5 pitchers include:

• Cliff Lee (except 2007)
• Tim Lincecum
• Trevor Hoffman from 2002-2007 (currently type 0)
• Tom Glavine
• Greg Maddux

Type 6: Slider (no backup)

(32% of pitcher seasons)

Type 6 pitchers are your prototypical fastball-slider flame throwers. A typical type 6 pitcher throws 66% fastballs and 24% sliders, with 5% change-ups and no curveballs. Type 6 pitchers are similar to type 2 pitchers, but they throw fewer secondary pitches. When a slider-throwing type 6 pitcher starts throwing a secondary pitch, be becomes a type 2 and vice versa.

Type 6 pitchers are the most common pitcher in MLB today, representing 32% of pitcher seasons from 2002-2009. They are the most common relief pitchers in baseball, but some successful starting pitchers were also type 6 pitchers. Type 6 pitchers often become type 2 pitchers, but here are some successful type 6 pitchers who have remained so throughout their careers:

• Randy Johnson
• BJ Ryan
• Joe Nathan
• Brad Lidge
• Josh Johnson 

Type 7: Splitter (Slider backup)

(4% of pitcher seasons)

Type 7 pitchers have become a rare breed, still I felt compelled to force a category for them. Splitter throwers have been some of the most successful pitchers of the last 20 years, so even if there are very few left in 2010, they are still worth separating them from the type 6 pitchers that they most resemble. A typical type 7 pitcher throws 60% fastballs, 15% splitters, and 10% sliders. 

Successful type 7 pitchers include:

(4% of pitcher seasons)

• Roger Clemens
• Curt Schilling
• Hideo Nomo
• Armando Benitez

Of course, Bruce Sutter is most famous for "inventing" the splitter. His version was thrown very hard, as a fastball with movement. Pitchers before him threw a split-finger pitch called a forkball, but that was more of a change-up. For some reason, the splitter has lost its popularity in the last few years, despite some of the best pitchers in recent history being avid proponents.

Typical type 7 pitchers throw even harder (91.5 mph average fastball) than typical type 6 slider-throwers (91.1 mph average fastball). This suggests that only hard-throwing slider-type pitcher ever become splitter-throwers. For some reason, recently very few type 6 pitchers have attempted this transition. There are probably good baseball reasons why this is has been the case, but I don't know what those might be. I'm an expert on machine-learning classification, not pitcher mechanics!

Hopefully the classifications and examples sound intuitive & intriguing. FWIW, here is a list of all 2009 pitcher seasons (over 20IP) with type classification (seasons ranked by pitcher VORP).

Despite the fact that these classifications are made without any consideration for performance, there are definitely differences in average performance between the eight categories of pitchers. I will follow up to discuss some of those differences. However, I think the most useful aspect of my categories is their descriptive quality. Although they rely (almost exclusively) on comparing % breakdowns of pitches thrown, the categoires can be described simply and intuitively. Not all pitchers fit neatly into a category, but very few pitchers don't belong in any category at all. I have made simplifications by ignoring knuckleballs and different fastball types, but without these simplifications, the list of categories might be even longer. My aim is to be insightful, but simple. 

I'm working on a site to host some of my data, including profiles for all pitchers, and their classifications by pitcher type. However, this will take some time. If you are interested in the type breakdown for a specific pitcher, take a look at this list, or contact me! 

Pitcher injury effects on projected value (Part I: Overview)

I often read about pitchers having high "injury risk," but what exactly does that mean? When projecting pitcher performance (for fantasy or otherwise), should I be concerned that a pitcher will miss time with injury, or that he will be ineffective when he pitches? I will try answer this question.

Of course, injuries are different, and even the same injury may have different effects on different pitchers. Just bear with me on this one. If modeling the effects of injuries on pitcher performance was easy, it would be less interesting.

Measuring Pitcher Value

If a pitcher's value is defined as runs saved, then his value can can be expressed as:

Runs saved  = Innings pitched (IP) * Runs saved per inning (ERA - replacement ERA)

When we look at past performance, we may want to consider the pitcher's defense, park factor and run context. However if we are projecting future value, these issues can easily be ignored. I am projecting ERA before park and defense adjustments. Other factors effecting ERA are so small compared to variance in projections that these factors can easily be ignored.

A commonly used statistic to measure pitcher runs saved is VORP. Therefore in this study, I project VORP from my IP and ERA projections. A lot of very smart people (such as Tom Tango) have argued very convincingly that VORP sets the replacement ERA level way too low. They are right, but for this study, setting the replacement level for ERA is not very important, as long as it is consistent. FWIW, VORP sets the replacement level ERA to around 5.4 (although it does so via RA). If we set the replacement ERA level to something like 4.9 for relievers and 5.3 for starters (where I think it belongs), there would not be much of a difference to this study.

In any case, I am trying to see how injury data can help me predict VORP, both by changing IP projections, and by changing ERA projections.

Data Sources & Results

I am using rich injury data from Corey Dawkins's injury tool. Using this data, I generate features like "did he have surgery in year X," "did he hit the DL in camp," and "how many days did he spend on the DL."

I have written about using this data to improve IP projection. Executive summary: there are several (7) categories of injuries that have large predictive effects on IP. 

More recently, I looked at how injury data can improve my FIP projections. Some features were useful, but not nearly as many features as for IP projections, nor by as much. I will write more about the details of this work later. I project ERA by translating my FIP projections, so I will write about the two interchangeably. 

Now that I have IP and ERA projections, both with and without injury features, I can compute four different versions of the VORP (runs saved) projection, and see which one is best at predicting actual VORP. Since pitchers without an injury history in Corey's database will not have their IP or ERA projection affected at all, I only include the pitchers affected by injury history. (These are all pitcher seasons 2005-2009 for which the pitcher either pitched or pitched the previous season & didn't retire. I'm trying to avoid selection bias by not ignoring projections for 0 IP seasons, or for little-used pitchers.)

	Correlation to (real) VORP	STDEV from baseline projection
Basic (no injuries)	0.587	0.0
ERA with injuries	0.599	1.2 VORP
IP with injuries	0.619	2.8 VORP
ERA & IP with injuries	0.624	3.2 VORP

Using injury data improves my ability to project both IP and ERA. However, IP changes are both more useful in predicting VORP, and they are also larger in the average effect on VORP.

None of this is surprising. If we know that a pitcher had Tommy John surgery last year, we should expect his IP to drop (usually to 0 IP) the next season. Also, we can project his IP to recover the season afterward to a higher than he would be otherwise projected. However, how should we expect his ERA to change when he comes back? It's hard to say.

Why injuries don't affect ERA/FIP projections much

Even for injuries that do not typically lead to a missed season, the effects on ERA are harder to predict than the effects on IP. Whereas there were 35 individual injury features that affect the IP projections, only 5 injury features had any effect on FIP projection (that my model was able to pick up). Here are those features:

-0.4223 * inj_elbow_surg_not_tj_2008

+ 0.0671 * inj_anyDL_2008

+ 0.1826 * inj_dl_camp_2008

+ 0.1641 * inj_anyDL_average

+ 0.2736 * inj_surgery_ip_average

Without completely explaining my notation, FIP projections increase if a pitcher was DL'ed last year, DL'ed in camp before the current season, or DL'ed in the previous three years. Recent non-Tommy John elbow surgery lowers the FIP projection, although any recent surgery increases the FIP projection.

The features are pretty non-specific, and the changes are not large (very few pitchers have their FIP projections affected by more than +- 0.3 FIP). Mostly, the injury features allow my model to reward pitchers who did not hit the DL or undergo surgeries in recent years. Contrast this to the extensive injury-based adjustments that I found for my IP model.

Selection bias?

I was surprised not to find stronger effects on FIP from injury features. Maybe there is a selection bias in the way I look for effects on FIP? After all, if a pitcher can't pitch well, maybe he will throw fewer innings and thus not be included in the data set?

For my FIP training, I include all pitcher seasons. However, I weigh instances by the inverse of expected FIP variance, estimated from real IP. I wrote about this in my previous post. The lower the pitcher's actual IP, the less weight I put on projecting his FIP correctly. However the effect for IP > 30 is small, and no pitcher seasons are completely excluded.

Still, to check for bias, I trained an FIP model on pitchers with 40+ IP, giving all of them equal training weight. Then I looked at the effects of injuries on this model. Nothing much changed.

Another possible manifestation of the selection bias might be that pitchers who are below-average to begin with can't handle an injury-based performance drop, and so pitchers who over-perform after injuries are over-represented? To test this possibility, I excluded all pitchers with a projected FIP > 4.2 (without considering injuries). Now I trained an injury model. Again, no noticeable changes.

Conclusions? Effects on Fantasy?

I am not going to claim that injury history can not help predict FIP/ERA. However, while it was fairly easy to find many injury features that have a significantly positive effect on predicting IP, that was not true for predicting FIP/ERA. Moreover, the features that do effect FIP/ERA projection tend to be fairly general, while some very specific features affect IP projection. I'd love to see someone else research this issue and come up with better results, but this is what I got.

If my observations prove to be true, what does this mean for projecting "injury risk" for fantasy pitchers?

Beyond the "he had Tommy John surgery, don't draft him," there are many cases where a pitcher has significant risk of missing time due to injury. The clearest example of this is for major shoulder injuries. Pitchers with shoulder-related DL stints are always at risk for another stint on the DL, whether or not they had surgery on the shoulder, and whether or not they were injured recently.

However, there is no noticeable effect on FIP/ERA from past shoulder injuries when projecting future performance based on past results. Therefore, if he plays, there is no statistical reason to expect a dip or to expect a rise in ERA based on a history of shoulder injuries. If you want an high injury-risk sleeper for your fantasy team, take a pitcher with a dodgy injury history, but strong recent performance when healthy.

However if a pitcher's recent performance wasn't good, don't think it will improve after surgery. Surgery is meant to get a player back on the field. Those effects are consistent enough to measure. However surgery does not typically improve pitcher performance. I am trying to find examples where is does, but I have yet to find any.

I will follow up with some specific examples, and with updated projections for 2010. I know the season has already started, but I think it's still early enough to make some of those projections interesting. 

How to weight FIP/ERA instances by IP

I like training on all of the available data, rather than only on data from "necessarily high IP sample size". However if I am trying to establish a relationship (between fastball speed and strikeout rate,  between FIP and ERA, or between past and future ERA, for example), I can not simply use samples based on 5 IP and 200 IP in the same way. When fitting a function to my data, I will invariably generate larger errors for those data points based on smaller sample sizes. Thankfully, I have a 60's era statistics book to help me out. According to M.G. Bulmer in Principles of Statisics:

If the form of the relationship between the variance of Y [dependent variable] and x [independent variable] is known, for example if the variance is known to be proportional to x, more efficient estimators can be obtained by weighting the observations with weights inversely proportional to their variances. In general, however, small departures from normality or homoscedasticity will have little effect on inference about the regression line and may be ignored.

In other words, if we can approximate the variance of our dependent variable by some function, then we can weight all data points in reverse proportion to that variance. But if the function does not suggest large differences in variance, we need not bother with the weighting.

Therefore, while many baseball studies are based on data points pruned by IP cutoffs, my studies use all available data, but with points weighted by the inverse of those points' variance (variance in whatever I am predicting), estimated based on the IP for the data point. This necessarily means that I have to compute a variance function every time I want to predict something new. Strikeout rates, ERA, FIP, etc all have different variance based on IP of the sample size. For every new dependent variable, I have to compute observed variance data points, plot them on a graph, and find a fit for this curve. However, this is no worse than picking (often arbitrarily, or worse, deliberately) an IP cutoff for one's baseball studies.

If you wanted to do a study predicting ERA or FIP from something (your independent variables are unimportant), you can take a shortcut by using my ERA and FIP variance estimates, based on single-season IP.

To approximate variance at each plotted data point (ie group of 320 pitcher seasons), I computed the RMS (root mean squared) error between real ERA (or FIP) and my basic projection system for ERA (or FIP). I could just use RMS deviation from the mean of each sample, but that would use incorrect baselines for the individual pitchers. My projection system is simple a statement of previous ERA/FIP, regressed toward the league average. Therefore, deviation from my simple estimate is a better measure of variance than is deviation from the mean of each sample.

Here are the data points for the observed ERA and FIP variance, along with my fits for that variance. The ERA and FIP variance is plotted on a logarithmic scale.

I fit both ERA and FIP variance to functions of the following type: (Also I minimize relative (rather than absolute) error at the data points, so as not to over-fit my function to the very low IP points.)

RA_VAR = A * (IP)^(B) + C

Here, C-term represents the ERA/FIP variance that one might expect from a very high IP sample, while the first term represents the variance differences between different IP sample sizes.

The variance functions for ERA/FIP are:

FIP_VAR = 296 * IP^(-1.63) + 0.32

ERA_VAR = 994 * (IP)^(-1.66) + 0.58

Since both functions have the same B term, it looks like variance for FIP and ERA converges at the same rate. Variance for ERA is always higher, which should surprise no one, but it's interesting that FIP does not converge faster than ERA, since it eliminates luck based on BABIP and other defense-related factors.

In any case, since the variance estimates converge at similar rates, M.G. Bulmer's book tells us that they can be used interchangeably without any effect on inference.

Therefore, if you are doing a study that looks at the effects of *something* on pitcher season ERA or FIP, you can use the FIP_VAR formula above to properly weight your data points, in order to compensate for variance caused by differences in IP sample size.

To get a feel for this function, here is a chart of suggested weights for various IP samples:

IP	1.0 / FIP_VAR
5	0.05
10	0.14
20	0.39
40	0.96
80	1.81
160	2.53
240	2.79

FIP & ERA baselines from projected IP (an alternative take on replacement level)

In my last article, I complained that my FIP/ERA projection system tends to regress all pitchers to the same baseline (around 4.6 FIP/ERA). This is an appropriate MLB average, but fringe pitchers (especially starters) should probably be regressed to a much lower baseline. So here, I show what such a baseline might look like. Incidentally, this can also be used to compute the "replacement level" FIP/ERA for starters and relievers.

The question I set out to answer was: given an IP projection (split by starter IP and reliever IP), what FIP/ERA should I expect from a pitcher? If I map actual IP to ERA, then I get a very nice graph with the properties that one would expect. But this graph is biased by the survivor effect. Better pitchers throw more innings, even if they start out with lower expectations.

Instead, what if we graph expected IP to actual FIP/ERA? Now we can answer a questions like: "what FIP/ERA should a team expect from a fringe starter (projected for 30.0 IP as a starter, or about five starts)?" My IP projection system is trained on all apitcher seasons from 2005-2009, including low-IP, high-IP and 0 IP seasons, so it projects realistic IP for all pitchers, not just the good ones. Also it gives separate estimates for starter IP and reliever IP.

Using actual performance for all pitcher seasons, I separated the pitchers into two groups:

1. IP >= 1 and (starter IP) >= 40% * (total IP)
2. IP >= 1 and (starter IP) < 40% * (total IP)

This is my categorization into "mostly starters" and "mostly relievers." The cutoff might seem arbitrary, but it separates the starter and relievers quite well. I could have left out a batch of pitchers around 50%, but I don't like excluding examples from my training sets, and there are not many such pitchers in any case.

Now I rank "mostly starters" by projected starter IP, and I rank "most relievers" by projected reliever IP. Within each group of 320+ pitcher seasons, I find the median FIP and raw ERA. Thus I create the series that are mapped below:

If some of that is confusing, let me explain that again with an example. Take the "mostly starter" IP series. The highest IP datapoint occurs at IP = 165.4. That is the median projected IP_Start for the top 320 pitchers seasons, ranked by projected IP_Start, provided that those pitchers threw at least 1 IP, and that 40% of their IP came as starters. These pitcher seasons include:

• Johan Santana (2009), projected to throw 214 IP (threw 166.2 IP)
• Brandon Webb (2009), projected to throw 206 IP (althrough he only threw 4.0 IP)
• Does not include Ben Sheets (2009), since he threw 0 IP.

I hope that makes things more clear.

Within a group of 320+ pitcher seasons (I use larger samples at the lower IP data points), I computed the median FIP and ERA, regardless of the IP for each instance. So in the example above, Johan Santana's ERA based on 166.2 IP in 2009 would be used on the same scale as Brandon Webb's ERA based on 4.0 IP in 2009. I purposely don't weight the instances by IP, since that would introduce survivor bias. Without biasing myself toward how many innings the pitchers actually ended up throwing, I want to know: given a projection of "X IP_Start" and "Y IP_Relief", what is a baseline for that pitcher's ERA and FIP.

Replacement Level

Incidentally, my graph also suggests possible replacement levels for starters and relievers. If we view replacement level as the level of performance that can be easily acquired from the waiver wire or from the minor leagues, then the low-end FIP/ERA projections from the graph should offer some guidance.

For relief pitching, the median FIP for low-end projections is around 4.5 (ERA 4.6-4.7). For starting pitching, the median FIP on the low-end is around 4.8 FIP, but the median ERA is around 5.3.

The low-end starter group might look like an outlier, but the median FIP/ERA are based on 400 pitcher seasons with the lowest IP_Start projections, but for those who actually pitched mostly as starters. This group had an average actual IP of 58.9 (52.0 IP as starters). The median actual IP was 42.7 (34.5 as starters). Therefore the group is a good representative of pitchers who one would not have expected to start many innings, but were pressed into starter roles and typically started multiple games. I believe they represent a good estimate of the kind of production a team might get from a spot starter pulled from the bullpen, or from a starter pulled up from AAA.

Going forward, I will use assume the following FIP and ERA (league-neural and park-neutral) replacement levels to fill a team's "missing innings" in projecting overall team ERA and overall pitcher VORP:

	FIP	raw ERA
Starter	4.9	5.3
Reliever	4.5	4.6

This is not the only way to estimate replacement level for pitchers, but these are the values most consistent with my individual projections. If one were to use a different system to project IP, then one would get different results. However I don't know of another system that accurately projects IP_Start and IP_Relief for low-end pitchers. Compared to my system, PECOTA and CHONE massively over-estimate the IP for low-end pitchers, especially rookies.

FIP vs ERA disparity

Since FIP is meant to predict ERA (after removing the differences due to defense and BABIP luck), it may seem strange that replacement starter ERA is 0.4 runs higher than replacement starter FIP. However students of DIPS will know that FIP tends to under-estimate ERA for bad pitchers, and over-estimate ERA for good pitchers.

My graph seems to suggest that FIP trails ERA nicely in the range (4.1, 4.7), but the relationship starts to break down beyond that range. This is (in part) because FIP assumes that:

• pitcher skills are limited to strikeout rate, walk rate and home run rate
• these skills are linearly related to ERA

Both of these relationships break down on the high end and the low end of pitcher performance. Elite pitchers tend to have lower BABIPs than do average pitchers (although luck and defense constitute most of the BABIP difference for individual cases). Also elite pitchers tend to be better than average at secondary skills like holding runners, situational pitching, and fielding their position. Conversely, low-end pitchers are worse than average at all of these skills. Also, since outs have a non-linear relationship with runs (the more outs a pitcher produces, the less valuable each extra out is), pitchers who get very few easy outs (strikeouts, popups or soft ground balls) tend to have an even higher ERAs than can be linearly approximated from the factors of FIP. Think of Adam Eaton of 2007-2009. His FIP and xFIP were bad, but his ERA was consistently even worse.

Effects on Team Pitching Projections

Armed with new replacement levels for starters and relievers, I should have better team pitching projections soon. Since there is a large separation between replacement level for starter ERA and reliever ERA, teams will suffer disproportionately depending on whether their "missing innings" (ie those innings not filled by IP projections for pitchers on their opening day roster) will need to be starter of reliever innings. The Nationals, with holes in their rotation, will have to fill those missing innings at a higher ERA than the Royals, who have a set rotation, but will need to fill some of their bullpen at replacement level.

Teams will get no credit for relievers projected to post an ERA above 4.9, but will get credit for any starters with projected ERA below 5.3 (before league and park adjustments). This will make my projections much more accurate, even if they are now being made a little too late to count as pre-season predictions.

Davis, Buehrle, Feliz and Mariano Rivera

Also the baselines help me to resolve a couple of specific problems I noticed for individual pitchers. I projected Wade Davis at a lower FIP and ERA than Mark Buehrle. Davis pitched well in 36 IP as a rookie in 2009, and his ERA, FIP and xFIP were all better than Buehrle. However there is no way that one should project him to be better than Mark Buehrle in 2010. The baseline FIP/ERA for starters by projected IP allow me to fix this problem. In the new FIP and ERA projections, I am regressing pitchers to their individual baselines, rather than to the MLB baseline of 4.6. This will help Mark Buehrle.

	IP Start (projected)	IP Relief (projected)	FIP baseline	ERA baseline
Wade Davis	90.5	9.4	4.69	4.78
Mark Buehrle	178.0	0.9	4.16	4.16

Similarly, I projected Neftali Feliz to post a lower FIP & ERA than Mariano Rivera in 2010. This is even more unreasonable, and new baselines should fix this:

	IP Start (projected)	IP Relief (projected)	FIP baseline	ERA baseline
Neftali Feliz	11.2	28.7	4.57	4.73
Mariano Rivera	8.9	55.5	4.13	3.93

Once I iron out a few more kinks, I should have new FIP, ERA and VORP projections for both individuals and teams. I have not yet done much with park adjustments, other than to adjust the individual and "missing innings" ERA projections to the team's park factor from 2009. It would be nice to consider a pitcher's park factor in terms of specific effects on HR rate, but everything that I've read on this issue seems to suggest that park HR factors vary too much year to year to be of much use. With so many teams having changed stadiums in the past few years (or having changed major characteristics of the field, wind patterns or the ball itself), long-term park factors do not seem very useful for predicting future park factors. I'd rather use a cruder park factor that is more current.