John Henry's comments on analytics concerning, but not condemning

The story of how Theo Epstein put together the 2004 team that finally brought a World Series to Boston again is, by now, common knowledge. To condense thousands upon thousands of words into just one: analytics. Early adopters of what has now become the standard in front offices across the league, the Red Sox ran the numbers to find what players did that contributed to on-field success that other teams weren't properly valuing. It won them a World Series, and then another.

In 2013, Epstein-disciple Ben Cherington made it three on the back of an offseason that would see the team forego the flashy moves in favor of a few smaller deals that had critics doubtful the organization would be able to bounce back from a dreadful 2012. Between that and all their past success, then, it was a bit surprising to hear John Henry say Wednesday that the Red Sox would be taking a step back from analytics due to the poor results of the previous four seasons: Via Nick Cafardo of the Boston Globe:

"I spent at least two months sort of looking under the hood, and came to the conclusion that we needed to make changes," said Henry, who also owns the Globe. "One of the things that we've done — and I'm fully accountable for this — is we have perhaps overly relied on numbers, and there were a whole host of things.

...

"Perhaps there was too much reliance on past performance and trying to project future performance. That obviously hasn't worked in three of the last four years."

Asked how he came to this conclusion, Henry simply said, "Results."

Certainly, recent years have provided three losing seasons in four. That the fourth provided a World Series doesn't completely erase that fact, though it should mitigate it. A careful observer would note, however, that the first of those four years saw an 89-win team (which likely makes the playoffs under the new system) demolished by a combination of sudden budget constraints and a heinous managerial choice by ownership which caused the team to self-destruct. And that of the worst (presumably analytic-driven) acquisitions in recent memory, most of them have a single year of data damning them. Not all Sox fans might believe it, but the book on Hanley, Sandoval, and Porcello have hardly closed.

Still, there are few who would have a better view into Boston's decision-making process than Henry, and it's entirely possible changes were needed. The question is whether the Red Sox are taking a step back from their analytics process, or a step away from it.

Moving away from analytics is strictly wrong, I would argue. Thanks to its nature as a game of a thousand instances--one pitch, one at bat, one ball in play--with individuals' results often effected only by themselves, luck, and an opposing player whose quality should even out to league average over time, baseball is the perfect sport for dissection. Few others are so tailor-made to be understood with numbers, and moving away from what is likely the deepest mine of information is foolish.

>

But no baseball statistician will tell you their model is perfect at telling you anything other than what their model measures. UZR is a perfect indicator of only what a player's UZR is, not their defensive prowess, the number of runs they saved in the field, or their likelihood of doing it again next season. Statisticians and their metrics are fallible and flawed, and if a team tunnels in too hard on them, they can lead any organization astray. When your preferred metric says player X would be an amazing signing in the face of all common sense evidence, it might be time to look at your metric rather than draw up a contract.

And if that's what John Henry is talking about, that's great! Statistics should be treated with a healthy skepticism particularly when a team has vast scouting resources available. But these failures of the last four years should be a message to re-evaluate the team's analytics department. That something needs fixing, rather than to be removed from the equation. That there may not be a perfect metric out there doesn't mean teams shouldn't be trying to find it, and getting closer to it, and using the imperfect models they develop en route. It just means they shouldn't take as gospel what they know to ultimately be (well-informed) conjecture.

The good news is that this is likely the direction the Red Sox are headed. You don't exactly hire Brian Bannister to go through your pitching staff with a fine-tooth comb if you're looking to ditch analytics. If Bannister's combination of on-field experience and analytical thinking is the model for the organization going forward, then all the better. Henry's later comments--that he's "an analytics guy," and that the Red Sox are looking for a "balance" and taking a "broader approach" are encouraging as well.

Just so long as in relying less on analytics, the Red Sox aren't investing any less into getting them right. That it's not easy to find that edge anymore and risky to try to push in on an uncertain discovery makes it that much more cost-effective to spend the time and money to get it right. If they reduce their investment commensurate with their reduction in emphasis, it'll be a fast way to make what attention they do give to analytics that much more dangerous than even the greater focus they placed on the numbers in the past.