How good (or bad) is the Red Sox bullpen, really?

The Red Sox were rained out on Tuesday which gave us all some time to collectively sit down and find something to complain about with our 106-win Red Sox team. Sure, the previous night the team had just broken a 106-year-old franchise record for wins in a season, but what about that bullpen? Baseball-Reference sent Red Sox Nation into a tizzy when it tweeted out that the Red Sox had the best bullpen WAR in the MLB at 4.3 bWAR. For the last six months we have been talking about the Red Sox bullpen being a potential fatal flaw of this spectacular team. Whatever drugs the B-Ref boys were on when they calculated this must have been potent.

I have watched the majority of the baseball that got the team to where they are and I for one agree with the people—this does not feel like a number one unit in all of baseball. The first thing I did to investigate was go on to B-Ref and check out how they calculate their version of WAR—bWAR. For a pitcher, bWAR starts with runs allowed and innings pitched, it then calculates a stat called x Runs Allowed (xRA) which measures for quality of offense being faced, park factors, team defense, and inter-league play, specifically DH or not. They then add an adjustment for starters vs relievers since relievers come in for short stints throwing gas and finally they adjust for leverage situations. All of this is mixed in a cocktail with their determination of what a replacement level reliever is and is spit out into their calculation.

This all seems logical to me. However, if you want to read it for yourself you may do so here and also you may compare all the different WAR calculations here. Just knowing how they arrived at this idea that the Red Sox had the best unit in the game wasn’t enough, I wanted to see how they compared using key stats from Fangraphs and Baseball Prospectus as well. The twitter account for Baseball Reference did claim that their bullpen war calculation did not account for park or leverage for some reason. This could matter quite a bit.

As you can see above Fangraphs, who begins their pitcher WAR stat by starting with FIP rather than runs allowed, feels much more on point placing the unit at 8^th best in baseball with an fWAR of 4.9. The unit also rated 8^th in K-BB rate and 7^th in batting average against—these are stats I think of as hugely important for a unit routinely tasked with high leverage end of game situations. The Red Sox unit also has the fifth lowest HR/FB ratio in the game with just 10.8% of their fly balls turn into home runs. Lastly, they are fifth best in baseball at limiting hard contact at 30.2%.

I then turned one of my favorite pitching statistics, deserved run average, over at Baseball Prospectus. This statistic might be the best measure available to the public at determining how well a player has pitched, though it is worth noting that this stat is not predictive. Like ERA, it is a descriptive stat that looks at the past rather than ahead. DRA did not like the Red Sox bullpen much at all placing them 14^th in baseball with a mark of 4.36—this is comparable to ERA. This mark puts the bullpen firmly below their AL competition with Houston 1^st, New York 3^rd, Oakland 6^th, and Cleveland 8^th. This feels accurate to me based on what my eyes have told me this season. This is an average unit.

There is no denying that when everyone is on this can be a unit that flashes above average. Craig Kimbrel can be the best reliever in baseball at times. We have collectively written so much about Matt Barnes this year that you already know how dangerous he can be. Joe Kelly for all of his inconsistency occasionally looks unhittable and occasionally looks, well, Extremely Hittable. Ryan Brasier has been one of the better stories of the year and throws 97 for strikes. I could go on and on here, but consistency is where this unit fails.

Overall, I think Fangraphs comes closest to capturing the essence of what this unit actually is—an upper-third, but not elite, bullpen over the course of the season. The unit has talent, but a lack of consistency keeps them from feeling like a true team strength. It must also be pointed out that the unit that we see make the roster for ALDS might be significantly different than the pitchers that were relied upon all year and drove the fWAR of the bullpen. Stalwarts like Heath Hembree and Joe Kelly remain very much on the bubble, for example, and a starter like Eduardo Rodriguez could join the ranks.

Tell me if you’ve seen this written in the last few days, but the narrative for baseball is that the regular season is a much more accurate picture of a team’s true talent than the postseason. A sport like this ought to be measured over months of data points not a five or seven game sample size. Unfortunately, or perhaps fortunately, the playoffs are what they are and even a mediocre unit can have an outstanding stretch when it counts the most. We know this isn’t the most talented group in the AL, but we also know it isn’t the worst. These guys are fine with a wide range of possible outcomes over a relatively small sample.

The outcome of these playoffs might come down to a big home run off Kimbrel or Barnes but perhaps it won’t. Maybe the story will be Chris Sale and David Price carrying the team to the World Series like Randy Johnson and Curt Schilling in 2001. Maybe the narrative is J.D. Martinez, Mookie Betts, and Xander Bogaerts mashing their way through the playoffs like the Houston Astros did last year. We don’t know, but what we do know is that this single thing, whether you believe it’s a weakness or strength, isn’t going to be the only reason they go all the way or go home. It will play a role, though.