2010 IP projections

Based on the simple ML system that I wrote about in my previous post, here are my 2010 innings pitched projections. I offer predictions for IP (broken down by starter innings and reliever innings) for all pitchers active in 2009, as well as for those who missed 2009, but had over 120 IP in 2008 (ie Ben Sheets, Mike Mussina, etc). The pitchers are ordered by 2009 IP.

I also offer my predictions from an earlier system that does not account for pitchers’ value stats (wins, VORP, saves, etc). I include the difference between value-drive and non value-driven projections in the last row. These differences range between +30 IP to -30 IP.  The IP_Start model rewards both 2009 value and previous value, while the IP_Relief model is more heavily weighted toward 2009 value. Roy Halladay and Johan Santana gain the most among high-impact starters, while Mariano Rivera and Joe Nathan gain most among high-impact relievers. More on that later.

It would be in poor form for me to include PECOTA or CHONE predictions next to mine, so let me share a few averages with you instead. In this context, rookies are pitchers who had at least 1/3 IP in 2009, but less than 50 IP in their careers. Projections for rookies with no MLB experience are not included. My system does not use minor league stats or scouting reports, so all such rookies will project to about 20-30 IP, depending on age.

	Actual  2009	My system (2010)	PECOTA (2010)	CHONE (2010)
Mean IP	65.1 IP	58.2 IP	99.9 IP	91.2 IP
Mean IP non-rookies	78.2 IP	66.4 IP	103.6 IP	95.1 IP
Mean IP rookies	14.8 IP	26.4 IP	86.0 IP	76.4 IP

My system expects an average pitcher to regress by 7 IP from 2009. Since total innings must add up, the roughly 500 * 7 = 3,500 missing IP will be made up for by (yet unknown) pitchers without any major league experience. That’s about 100 IP per team.

In my system, veteran pitchers will regress by about 12 IP from 2009 on average (including pitchers who retire; more on that later). Rookies who’ve had a cup of coffee in the majors  will expect to pitch almost 12 IP more in 2010 than in 2009. All of this is pretty consistent with the averages that I outlined in the previous article.

However CHONE and PECOTA predictions are not consistent with these averages. In either system, the average pitcher will increase his innings pitched from 2009 by over 30 IP.  The average rookie will increase his innings pitched by over 60 IP! The average major league rookie is projected to throw more innings than Mariano Rivera.

I don’t think that any of this is realistic. CHONE and PECOTA allocate over 100% of 2010 innings to veterans, then allocate a further 20% of 2010 innings to rookies, and none of this accounts for pitchers making their major league debuts this year (while my system leaves 100 IP per team for those pitchers).

Some may argue that PECOTA and CHONE make “if he makes the majors” projections. A lot of these innings will end up getting pitched in the minors, and major league innings can be adjusted on a team-by-team basis. I think BP has a manual process where pitchers are selected based on likely playing time, and PECOTA IP projections are adjusted accordingly.

However I’m not sure that such a process is necessary, nor do I think it’s optimal for projecting team pitching totals. As I showed in my last posts, there are pitchers every year that come out of nowhere to pitch significant innings. Also the top prospects, as a group, collectively pitch fewer innings than CHONE or PECOTA would lead you to believe. Lastly, there was a nice article on BP recently (by Tommy Bennett, behind the pay wall) showing that a team’s 5^th and 6^th starters pitch comparatively similar innings on most teams. Therefore picking which players will be in the rotation, or who will be on the 25-man roster, is both futile and also counterproductive for projecting individual IP.

It would be better to compute an independent set of IP projections that respect the recent averages, and then (possibly) make some small team-based adjustments by hand or automatically. It’s more realistic to say that the Yankees will use 20 pitchers in 2010 whose collective IP is slightly less than the Yankees’ overall 2010 IP, rather than listing their top 10 or 13 pitchers, whose collective IP adds up to or exceeds the Yankees’ overall IP.  (FWIW, the Yankees used 24 pitchers in 2009.) Now using projected ERA or another value rate stat and a realistic IP projections, it should be possible to approximate the Yankees’ overall projected pitching value in 2010 in a robust manner.

Hopefully I’ll have time to complete all (most?) of those steps before the season gets under way.

Although my system is much better than CHONE or PECOTA at generating IP estimates for rookies, and somewhat better at projecting veterans, it is still far from ideal. Most notably, I think my (non)handling of retired pitchers hurts my system’s ability to project veteran starters’ IP rationally.

I think it’s important to train a system with examples where veterans don’t come back strong after a bad or injury-plagued year, and also to include examples of sudden falls to 0 IP after decent seasons. Both these things do happen to pitchers and the model need to take that into account. However, I think my system is overdoing the downward regression bit for starting pitchers. The only pitcher who threw 130 IP in 2009 who is expected to increase his total in 2010 is Johan Santana (who is expected to move from 166.7 IP to 174.0 IP).  A typical starter who threw 180 IP in 2009 with solid results (Roy Oswalt or Joe Saunders) is expected to throw 30 IP less in 2010. This sounds a bit harsh to me.

The cases of voluntary retirement for guys throwing over 100 IP in the previous season are few, but they might be screwing the results for all starting pitchers in my model. Greg Maddux and Mike Mussina were projected for 140 IP and 180 IP respectively in 2009 by my system. Both retired and threw 0 IP. In a model designed to minimized root mean square error, a few such examples can make a significant impact. Both Mussina and Maddux retired well before the 2009 regular season. My model should have been privy to that information, and ascribed their severe downturn in IP to retirement, and not so much to other factors.

Ben Sheets and Jeff Francis were also projected to throw a bunch of innings in 2009, but missed all of last year due to injury. Their cases should absolutely be in included in the model. However before I can build a decent injury-based model for IP, I need to account properly for known preseason retirement. Now where can I get that damn retirement data…

Excessive exuberance for young pitchers? (III)

Now that I have showed that my "simple linear weights model" predicts IP better than CHONE and PECOTA, I should probably explain how my model works. However, I'm not sure how much anyone who's reading this cares about what features I used or how I avoided over-training the model. So I'll keep this brief, share a few links, and promise to have my 2010 IP projections up here soon. Maybe as early as tomorrow.

I trained my model in pieces. Most importantly, I trained separate models for IP_Start and IP_Relief. Here is a simple graph of what I did:



My basic inputs were:

• "IP stats" (such as IP, IP_Start, IP_Relief and also DNP (did not play), ROOKIE, etc) for the previous four years
• "Value stats" (such as wins, losses, holds, VORP, SNWP (support neutral winning percentage for starters), WXR (wins added in relief), etc

To get the final model, I first trained basic models for IP_total, IP_Start and IP_relief purely based on previous playing time. Then I trained the final IP_Relief model based on the basic projections and also the value stats. The final IP_Start model uses this same IP_Relief model, as well as value stats. This allows the final projection to be a simple sum of the IP_Relief and IP_Start models.

Each of the five trained models can been seen in a long text file here. Most of the features are self-explanatory, although a few are not. I'd be happy to explain what they mean, if anyone wants to know.

Note that "_2008" features simply mean features for the previous year. I named all the features as though I were training on 2009 pitcher seasons, although I actually use 2005-2009 pitcher seasons for training. Also "_average" features are averages for the previous three years not including the last one (ie "_2005", "_2006" and "_2007"). The "_ip_average" features (the few there are) are four-year averages, weighed by seasonal IP. So they are a sort of "how did he do when he actually played" features. They are very optimistic estimates, in other words. Most of the features used are "_2008" features, which shows just how much projections are dependent on very recent performance.

Again, I won't spend much time on these models. Each model reports a correlation with the data that it is trying to predict (based on 5x cross validation). The correlation is the "r" in "r^2" that you statistically-minded people know and love. My most basic IP_total projection system gets a 0.71 correlation with the data (using previous year's IP only, and not included here), while the final system claims a 0.76 correlation. That doesn't sound like much, although that's a 10% improvement in r^2.

More importantly, my final projections take account of a broader set of features than the more basic projections. Statistically speaking, there is not much gain from looking at value stats when projecting IP. We know that a pitcher who throws 200 IP and is very effective is more likely to throw 200 IP again than a guy with the same 200 IP who was at replacement level. However, the difference in ability is usually reflected in the previous usage anyway (replacement level pitchers don't often accumulate 200 IP), so the information gain is not large.

However I would rather use this information, even if the gain is small. I want my system to take account of the fact that Mariano Rivera provides lots of value, and thus project him differently than Brad Lidge, who had a similar number of innings pitched in relief. Even if performance and playing time is highly correlated, I would rather use more factors, as long as their relationship makes sense to me, and as long as I am not using too many.

People in AI often say that feature selection is an art, as well as science. If there is no information gain in a feature, my system won't use it. However, most of the relationships I use in my models are not statistically significant. Which features I keep and which ones I don't keep is a matter of discretion. Generally, I want to use features that make intuitive sense, and which comprise both "_2008" information, and information from previous years. I could train a model with almost the same accuracy using only "_2008" features, but it would consider Brandon Webb and Ben Sheets as old rookies going into 2010. We know that's not the case, so I'd rather have a model that as least tries to use all four years' worth of data.

As it turns out, older data is useful to predict starters' innings, but is almost useless for predicting future relief innings. Here is the final IP_Relief model:

IP_Relief_2009 = 

0.4164 * L_2008 

+ 0.3454 * BS_2008 

+ 0.3444 * VORP_2008 

+ 0.2825 * Hold_2008 

- 4.9235 * SNVAR_2008 

+ 1.9615 * WXR_2008 

- 3.3587 * bin_change_LG_2008 

+ 0.1924 * SNWP_R_2008 

- 0.3574 * W_average 

+ 0.4407 * L_average 

+ 0.078 * VORP_average 

- 0.9571 * SNVAR_average 

+ 0.8097 * LG_average 

+ 0.1252 * proj_IP_v26 

- 0.1387 * proj_IP_Start_v26 

+ 0.4968 * proj_IP_Relief_v26 

+ 5.1194

You don't need to understand what all these features mean to see that next year's relief innings are heavily dependent on WHYDFML. Total value added (with value added for starting pitching taken out) is responsible for 1/2 of relief innings pitched prediction, while only 1/2 is dependent on the simple IP_Relief model that accounts for previous years' innings pitched. Therefore an effective reliever who pitched 40 innings in 2009 might be projected for 60 IP in 2010, while a reliever who throws 80 IP every year might be projected for less than 60 IP next year, if his last year wasn't so good. Having been an effective reliever in 2007 means nothing, if your 2008 sucked.

I'm not sure why SV (saves) did not make the cut for features used to predict IP_Relief, while Holds, BS (blown saves) and WXR (wins added in relief) proved to be useful indicators. Maybe this goes to show that saves statistics get abused too much nowadays. A setup man with twice the holds and double the WXR as another setup man is most certainly a more valuable setup man, but a pitcher with 60 saves is not inherently more valuable than a pitcher with 30 saves.

The IP_Start also has an interesting caveat:

IP_Start_2009 = 

0.8554 * W_2008 

+ 0.877 * VORP_2008 

- 16.6386 * SNVAR_2008 

+ 2.8454 * SNWP_R_2008 

- 0.606 * L_average 

+ 0.5458 * VORP_average 

- 2.2624 * SNVAR_average 

+ 0.2685 * proj_IP_v26 

+ 0.6921 * proj_IP_Start_v26 

- 0.761 * proj_IP_Relief_v26_w_value 

+ 6.1707

Here "SNWP_R" is a BP feature (Support Neutral Winning Percentage) that I turned into an "over replacement" stat. This is BP's stat for measuring starter quality on a 0.0-1.0 scale where about 0.44 is replacement level (and 0.6 is really good). I smoothed values for low-IP guys and count the points above 0.44 (multiplied by 100). 

What's interesting is that SNVAR (wins added as a starter, also from BP) is negative. Positive values for VORP and negative ones for SNVAR mostly neutralize each other for starters, while SNWP_R is always positive. Therefore a starter who is really good in 160 innings will get the same credit as a starter who is equally good in 220 innings (same SNWP but higher SNVAR).

The basic proj_IP_Start_v26 (IP & rookie stats only) IP_Start model is essentially a regression of last year's IP_Start (with extra credit for previous years' IP), so it's interesting that the bonus for a good 160 IP pitcher is the same as that for a good 220 IP pitcher (although the later is hurt more by regression in the simple model). The model does not a have a feature to say "this guy has pitched 220 innings every year for five years, so I should give him credit for that skill," but still, it seems to suggest that such pitchers are more lucky than skillful. As Brendan Webb showed last year, being a workhorse for years does not preclude a regression to the mean. It's easy to say that a guy is special in retrospect, but projecting into the future, there is no reason to give any of today's pitchers a 50th precentile projection of 220 IP.

I'd be happy to discuss more details about these models if anyone is interested. Otherwise, I'll proceed to 2010 projections and thoughts for further improvements. Including a better understanding of how injuries affect IP prediction.

Also, I need to repeat these steps (and some other steps) for ERA prediction, and VORP prediction (although hopefully ERA and IP will interact well enough to produce VORP projections directly). 

Excessive exuberance for young pitchers? (II)

In my last post, I claimed that projection systems tend to over-estimate playing time for young pitchers. I went on to claim that a system that ignored minor league stats and treated all rookies equally would give better overall projections for pitchers’ playing time. As it happens, I’ve built such a system.

As I have written about before, I’ve been working on a pitcher season projection system for some time. I started building my projection system with data from Baseball Prospectus, which does not include comprehensive minor league stats. I’ve never bothered to acquire minor league stats elsewhere, so in my system, all I know about a rookie is that he is rookie, and nothing else. Actually, I know other things (skills ratings, Baseball America ratings, fastball velocity), but here I will demonstrate a simple projection system, that only uses a pitcher’s basic MLB stats, rookie status, and age.

I will explain my system later. For now, let’s just say it’s a linear-weights system based on the past four years’ stats such as:

• IP_Start, IP_Relief, rookie status & missed seasons
• Wins, losses, saves, hold, VORP (total runs saved), WXR (relief pitching value in expected wins), etc

I trained my system on pitcher seasons from 2005-2009 that either:

• Recorded at least 1/3 IP in the majors that season
• Recorded at least 1/3 IP in the previous season

Therefore, my system takes account of the fact that, each year, 20% of pitchers do not return to the big show.

Perhaps I should remove the voluntary retirement seasons. However most pitchers retire when they can no longer pitch in the majors, so the current system is reasonable. In any case, it is better than not accounting for the fact that some pitchers who had decent seasons go directly to zero the next year.

After building my system, I commit the sin of training and testing on the same data. However, I could not leave out 2009 for testing only. I am using very little data anyway (only five years’ worth). I could give you the testing data on clean 2004 data, and later I will, but I doubt you’d. Just trust me that the system was built in a way that reduces over-training and keeps things very simple. If you are interested in the details, please read my next post.

In any case, I compared the predictions of my system for 2009 IP with last year’s preseason projections from CHONE and PECOTA. I tested on all pitchers where my system, CHONE and PECOTA all provided predictions. This consisted of all pitchers who had 2008 or 2009 major league IP, sans about 40 pitchers that were missing in either PECOTA or CHONE.

None of the omissions were notable pitchers. Although I do think that it’s notable that projection systems that project huge numbers of minor leaguers (mostly with unrealistic stats) still manage to miss some pitchers who make it to the majors. I think this goes to show the point I made in the last post: we really don’t know which minor leaguers will get playing time, so we might as well just play the percentages. But I digress...

It would be in bad form to post PECOTA and CHONE projections here, since I do not own this data. Therefore, I will just summarize the results, and offer a few examples.

For the full set of 2009 pitchers, my average projection error is significantly lower than either PECOTA or CHONE. Also, my IP projection beat PECOTA head-to-head 63.8% of the time, and it beats CHONE 66.3% of the time. Most surprisingly, my projection is best out of all three 54.8% of the time!

	My system	PECOTA	CHONE
IP error STDev	43.6 IP	55.5 IP	61.6 IP
Best projection	54.8%	23.3%	22.2%
Mean error (IP)	-0.4 IP	25.8 IP	32.2 IP
Mean IP	54.0 IP	80.1 IP	86.6 IP

The last two rows here show just how much PECOTA and CHONE over-estimate playing time for an average pitcher. If we include O IP seasons for pitchers who played in 2008, the average pitcher in 2009 threw 54.4 innings. And yet, PECOTA projected the average pitchers for 80 innings, and CHONE for even more. These are not reasonable baselines (even if we removed the empty seasons).

You may argue that 0 IP is never a good projection for a major league pitcher. That is true. However, Ben Sheets, Tom Glavine and Kei Igawa all looked to have reasonable shots to pitch in the majors in 2009, and none of them did. I think my system does deserve credit for projecting lower totals for guys who don’t pitch at all.

Actually, my system still easily beats CHONE and PECOTA if we remove all 0 IP season. To make closer, I remove all seasons with <20 IP. With that, the average pitcher IP rises to 87.6, and all of the systems are equally effective:

	My system	PECOTA	CHONE
IP error STDev	45.9 IP	43.3 IP	47.0 IP
Best projection	34.1%	32.3%	33.8%
Mean error (IP)	-19.7 IP	2.9 IP	7.1 IP
Mean IP	67.9 IP	90.5 IP	94.7 IP

Ok, so my system does better because I project fewer inning for pitchers who retire, right? Not quite. Pitchers that both PECOTA and CHONE projected to throw over 100IP, but who threw less than 20 IP include:

• Ian Kennedy
• Robert Mosebach
• Mitch Atkins
• Andrew Carpenter
• Zach Johnson
• Brandon Webb

Chances are, you know who 1.5 of those pitchers are, and only if you're a golf fan. No projection system saw Brandon Webb’s season-ending injury. However, what were PECOTA and CHONE doing projecting over 100 IP for the misfits?

As I wrote in my last post, I think these systems over-value minor league stats, and give unrealistically high projections to young pitchers, and especially to rookies.

Consider how the projection systems perform if we keep 0 IP seasons, but remove all seasons pitched by rookies (less than 50 IP of prior MLB experience):

	My system	PECOTA	CHONE
IP error STDev	45.2 IP	52.0 IP	59.1 IP
Best projection	48.3%	25.7%	26.3%
Mean error (IP)	3.0 IP	18.0 IP	25.1 IP
Mean IP	67.8 IP	82.7 IP	89.8 IP

As you can see, my system is still best (since CHONE and PECOTA over-estimate non-rookie pitcher IP also), but not nearly by as much as with rookies included.

Sure, these systems look great when they nail their projections for Jeff Niemann, David Price, Tommy Hanson, Brad Bergesen and Andrew Bailey. However they miss more often than they hit. If you say that Jeff Niemann and Tommy Hanson are better prospects than Ian Kennedy and Clay Bucholtz, it is precisely because they had better 2009s that you think so. Kennedy and Bucholtz were also highly rated prospects a couple years ago, and if they succeeded by now, we would have said “but of course I saw that coming.” And perhaps you would be right. However, as PECOTA and CHONE show, on average, these prospects struggle more than we think that they should.

Ironically, good projection systems that use minor league stats get crushed by a simple system that does not use minor league stats for precisely that reason. I’m not saying that minor league stats have no use. But we should not put too much stock in great minor league stats, since teams clearly don’t.

Excessive exuberance for young pitchers?

I don't like to criticize other peoples' work, but it seems to me that most projection systems for pitchers have a major flaw: they way over-estimate the IP (innings pitched) for young pitchers.

Consider a young pitcher that most of you have heard of. According to FanGraphs, here are the 2010 IP projections for Phil Hughes:

system:	projected:
Bill James	121.0
CHONE	116.0
Marcel	81.0
Fans	138.0

These numbers look reasonable, right? Hughes is a strong candidate for the Yankees #5 starter role after a solid minor league career and a good season in relief for the Bombers last year. And yet, what is a reasonable high-end estimate for Hughes’s IP this season? Fifth starters never throw 200 IP, and in any case, I doubt that the Yankees would work Hughes that hard even if they need to move him up in the rotation. Also note that he has never threw more than 160 innings as a professional.  A reasonable high-end projection for him would probably be 140-160 IP, if everything goes well. And what if it does not?

It’s hard to estimate a low-end case for Phil Hughes, but the lower bound for any pitcher is 0 IP. Hughes does not have a bad injury history, but all pitchers get hurt sometimes. Also, he might struggle and lose his rotation spot (assuming he is a starting pitcher this year).

I’m not saying that 120 IP is a bad estimate for Phil Hughes, but it’s a more optimistic projection that is might seem at first.

Some of you may disagree, and frankly I’m not sure that this particular projection is a bad one. What I am saying is that, collectively, IP projections for young pitchers are higher than is consistent with what really happens in the majors.

Consider CHONE IP predictions for all Yankees pitchers. Here are the starters, not including the top four (Sabathia, Vazquez, Pettitte & Burnett):

	Projected IP:
Phil Hughes	116.0
Chad Gaudin	162.0
Dustin Moseley	108.0
Sergio Mitre	101.0

Does anyone actually think that these four pitcher will collectively pitch 480 innings? There is no chance!

This problem is not an isolated case for the Yankees. Pretty much every team’s pitchers’ IP are over-estimated by CHONE, even if you count only pitchers who are sure to be on the Major League roster. The Orioles are projected to have seven pitchers with at least 18 starts and 100 IP each. Even if the season had enough games for 200 starts, there is not much chance that these exact seven pitchers will get all of those starts.

I think that this is the key to understanding why projections for young pitchers are historically too high (I will show just how high they are later). If you look at individual projections for young pitchers, they may seem reasonable, but collectively, they just don’t add up. This is because we really don’t know which young pitcher is going to get playing time, so we should severely discount the projections for all pitchers who are not established major leaguers. The IP projections for all pitchers on a team’s Major League roster should add up to significantly less than the total innings for that season. Instead, all the projection systems that I have looked at tend to have those stats add up to more than the total innings for that season.

Phil Hughes is a good pitcher, and will not get demoted quickly. I think projecting 120 IP for him this year is a bit high, but it’s not a unreasonable 50^th percentile projection. However, CHONE also projects 56 IP for Zack Segovia. Zack who? Exactly.  Might he pitch 56 innings for the Nationals this year? Maybe, but that is not a reasonable 50^th percentile projection, at least not for an impartial computer system.

I don’t mean to pick on CHONE. PECOTA has the same flaw. However PECOTA data is not public. I’ve heard that OLIVER has good projections, but I don’t think that those are available publicly either.

I don’t know how CHONE or PECOTA are trained, but I don’t think they take proper account of how innings are distributed between veterans and rookies. Over the past five years (2005-2009), here is how innings have been divvied up:

• 640 pitchers per season
• 226 pitchers qualify as rookies (less than 50 IP previously)
• 67.7 innings per pitcher
• 37.6 innings per rookie pitcher
• 66% of innings are thrown by starters
• 20% of innings are thrown by rookies

The short version is that, in today’s game, most innings go to experienced pitchers, especially in the starting rotation. Occasionally, there are pitchers like Tim Lincecum, Rich Porcello and Dontrelle Willis who rack up a large IP tallies as rookies. It’s nice if your projection system finds those guys. But in the meantime, that system will probably over-estimate playing time for many other good pitchers, who will struggle in the majors, or who will see their rotation spot or roster go to a veteran.

Both PECOTA and CHONE are very good projection systems, and they both take minor league stats into account. But I would argue that they would be better off with unsophisticated, lowball IP estimates for all young pitchers, until those pitchers establish major league records worth projecting off of.

In my next post, I will demonstrate such a system.

Why the Reds paid $30 million for Aroldis Chapman

This is not a post about money. I'm just going to demonstrate why a major league ready LHP with a huge fastball is automatically a valuable pitcher, regardless of his other qualities (except maybe health).

According to what I read, Chapman's fastball sits in the mid to low 90's and touches 98 mph. He keeps his velocity deep into games, so he projects as a starter. Let's say that he is able to to work his way up to sit at 95 mph, with below-average control for a starting pitcher.

According to my league-based models from the last post, Chapman's strikeout rate projects at 10.3 K/9 in the NL, and at 9.2 K/9 in the AL. This is baseline, so he might do better, he might do worse, but this is a 50th percentile estimate of sorts. Now let's give him poor walk rate for a starter. Let's say he is A.J. Burnett. So we project Chapman at 4.22 BB/9, second-worst for 200 IP starters in 2009. While we're at it, let's give him Burnett's home run rate from last year as well (1.09 HR/9). Now we can approximate Chapman's ERA using FIP, a simple fielding-independent ERA predictor:

(HR*13 + (BB + HBP - IBB)*3 - K*2) / IP + (annual league constant)

For the league constant, let's use Bronson Arroyo's 2009 constant of +3.21. Putting it all together, we get:

(1.09*13 + (4.22)*3 - 10.3*2) / 9 + (3.21) = 3.90 FIP

Therefore, a Chapman who sticks in the starting rotation, puts up a 50th-percentile strikeout rate, and has A.J. Burnett's control, is a very valuable pitcher. According to Dave Cameron's calculations, replacement level for NL starters was 5.37 FIP in 2008. Therefore, the projected Chapman, over 200 IP, saves (5.37 - 3.90) * 200/9 = 32.7 runs, as compared to a replacement-level starter (i.e. Micah Owings).

If we use the general rule that 10 runs = 1 win, then Chapman is a +3.3 win pitcher, even with below-average control and assuming his 50th-percentile strikeout projection. That's a lot of projection, but if the Reds think that Chapman is healthy and almost ready for the majors, then their signing makes a lot of sense.