More Moneyball

688484_baseballIn my previous post, I argued that Michael Lewis’s influential bestseller Moneyball, widely cited in academia, ultimately relies too much on hyperbole to make its claims about the superiority of Billy Beane’s statistical methods in managing the Oakland Athletics baseball team. In this post, I assess the value of those Moneyball methods by examining the results of Oakland’s 2002 draft, a central event glamorized in the book as a showcase of Beane’s “scientific selection of amateur baseball players.”

This is a long, baseball-heavy post, so let me cut to the chase at the outset. Oakland’s Moneyball draft of 2002 was not the smash success that Lewis’s book forecasted. The seven years since the draft bear that out. Beane had seven first-round picks that year but drafted none of the eleven all-stars signed from the 2002 draft, despite exercising almost twenty percent of all first-round choices in the draft. In fact, only half the players on Beane’s wish list of the top twenty players in the draft ended up playing even a game in the major leagues. To tell a compelling story of Beane’s superiority, Lewis overstates what Cass Sunstein and Richard Thaler call the “blunders and the confusions of those who run baseball teams.” It turns out that other teams do a pretty good job of identifying talent too. In the 2002 draft, Oakland did not draft the best players available, while other teams using traditional methods did equally well or better.

So, what is the lesson for Moneyball? Lewis describes how Beane imported quantitative methods from the academic and financial worlds to identify assets, in this case baseball players, undervalued by the market. The modest notion that statistical methods can be invaluable in the search for these undervalued assets, particularly in an industry so obsessively numbers-oriented as baseball, is unassailable. But Lewis’s claims that Moneyball demonstrates the outright superiority of Beane’s quantitative methods in identifying the best talent are much more difficult to sustain, and Moneyball does not convincingly establish the backwardness of other teams, which had already begun erasing whatever advantages Oakland possessed by the time of Moneyball’s publication.

Instead, Moneyball demonstrates a slightly but importantly different lesson about Oakland’s successes during the early 2000s.

So, the familiar Moneyball theme of Beane outsmarting the league by finding great talents invisible to other teams clouded by their prejudices is largely overstated by Lewis. Not even the most committed Moneyball devotee would argue that the heroes of the book—Scott Hatteberg, Chad Bradford, Ricardo Rincon, Jeremy Brown—were star players. They were role players, and it’s in finding role players at a bargain where Moneyball was most useful. The advantage of Moneyball methods was not that they were so superior per se to traditional approaches in identifying top talent, or that traditional approaches were as wrongheaded as Lewis depicts. Moneyball, by offering a different playbook than other teams followed at the time, rarely identified star players with any more accuracy than traditional methods, but instead represented an alternate strategy that helped Oakland maximize the return from the pile of cheaper, marginal players that other teams did not covet. Moneyball was most valuable for finding cheaper players, not better ones. Finding cheaper is different from finding better, and Moneyball gave Oakland an advantage mainly at the former rather than the latter.

The best way to explain this lesson is by carefully assessing Oakland’s 2002 June amateur draft. Lewis documents this draft minute-by-minute in the book to illustrate Beane’s methods. And for good reason. Oakland held seven first-round picks (including sandwich picks), and seven of the first thirty-nine picks overall—a historic opportunity for Beane and his team. However, judged seven years later, Oakland’s results from the draft appear unremarkable. Among Oakland’s seven first-rounders, only three had more than a cup of coffee in the major leagues—Nick Swisher, Joe Blanton, and Mark Teahen—though none is too much more than an average player and all three were traded away by Oakland. As a Bay Area sportswriter summarized, “Basically, Moneyball happened, and it didn’t change the course of the A’s. Nice story. Good book. Not a lot of history made by those seven players.”

My point is not that Oakland’s 2002 draft was a failure, as some suggest, or decisively undercuts Moneyball as baseball philosophy. Neither is quite right in my view. But Beane and the Oakland braintrust, at least in Lewis’s account, dramatically overrate ex ante the success of their draft. Oakland’s draft is not just good in their view at the time, but “unfair,” nearly otherworldly. “It counted as one of the happiest days of Billy Beane’s career,” according to Lewis, and Beane warned “[t]his doesn’t happen. Don’t think this is normal.”

Thanks to the book, we have Beane’s internal ranking of the top twenty players available for the 2002 draft and can judge Beane’s skills at player evaluation with rare insight. Beane’s list, as documented by Lewis, were the “twenty players they’d draft in a perfect world” if “money were no object and twenty-nine other teams were not also vying to draft the best amateur players.” As a first cut, only ten of the twenty players on Beane’s list ever appeared in even a regular major league game. Although the major league draft is a “f_cking crapshoot” in Beane’s words, his list’s 50 percent success rate is less than the 60 percent rate for first rounders from 2000 to 2005. When compared to the first twenty players selected in the 2002 draft (as opposed to the entire first round), Beane’s list looks even worse by comparison. Ninety percent, eighteen of the first twenty picks in the 2002 draft reached the majors, including five all-stars.

To compare Beane’s acumen against the rest of the majors yet more precisely, we can focus on only the twelve players on his list who were not first-round picks of, nor likely considered first-round talents by other teams—Ben Fritz, Stephen Obenchain, John McCurdy, Jeremy Brown, Bill Murphy, Steve Stanley, John Baker, Mark Kiger, Brian Stavinsky, Shaun Larkin, Brant Colamarino, Mark Teahen. None of these players were ranked among Baseball America’s top forty prospects in the 2002 draft, but Beane staked his draft on them by drafting all but one of them. Mark Teahen was a nice find who has enjoyed a solid if unspectacular career with Kansas City. John Baker, who was released by Oakland, eventually caught on with Florida. But unfortunately for Oakland fans, Beane selected Fritz, Obenchain, McCurdy, and Brown with first-round picks, and took Murphy, Stanley, Kiger, Stavinsky and Colamarino later. Those nine players combined to appear in just twenty-three major league games total among them over eight seasons.

Lewis carelessly attributes the other teams’ disinterest in players like Brown and Colamarino, as two examples, to an irrational prejudice that they “do not look the way a young baseball player is meant to look.” But many teams weren’t interested for the simpler, less interesting reason that Brown and Colamarino just weren’t very good. In the end, Lewis was wrong to crow about how smart Beane was not to share other teams’ bias against Jeremy Brown’s lack of athleticism and defensive ineptitude—the very characteristics that killed his major-league prospects. Jeremy Brown, cited by Ian Ayres and Sunstein and Thaler among others as the poster child for Moneyball’s value, washed out of baseball not long after the book’s publication.

One aspect of Oakland’s 2002 draft truly was not “normal”—the Athletics took a college player with their first 23 picks and drafted 25 collegians among their first 28 picks. The 2002 draft turned out to be an unfortunate moment to be so strongly biased against high school players. Eight of the eleven players signed from the richly talented 2002 draft who were eventually selected as a major-league all-star came right out of high school: Zack Greinke, Prince Fielder, Scott Kazmir, Cole Hamels, Matt Cain, Jonathan Broxton, Brian McCann, and Josh Johnson. All but Johnson (fourth round) were taken in the first two rounds. Only three players signed out of college from the 2002 draft have been chosen as an all-star so far—Joe Saunders (first round), Curtis Granderson (third round), and Russell Martin (seventeenth round). It is worthwhile to point out as well that not a single one of these eleven all-star players was on Beane’s list of the top twenty players in the 2002 draft. As Beane admitted to Baseball Prospectus in a recent interview, “That list would look very different today.”

Oakland’s focus on college players was clearly by design, as Lewis colorfully explained by reference to Beane’s view of one particular high schooler in the draft. Lewis noted that Beane had no interest in Scott Kazmir, a high school pitcher coveted by Detroit and Milwaukee (though drafted by the New York Mets). As Lewis explained, “[P]icking a high school pitcher like Kazmir is exactly the sort of not-so-bright decision both franchises had a knack for making.” Yet Kazmir was one of six high school pitchers from that draft to become a major league all-star, as well as one of the first 2002 draftees to reach the majors.

The cost of dismissing high school players illustrates a tension in Lewis’s presentation of the Moneyball philosophy. Lewis explains that the Moneyball insight is that “when you rule out an entire class of people from doing a job simply by their appearance, you are less likely to find the best person for the job.” However, Oakland ruled out as a practical matter an entire class of prospects from consideration in the draft—high schoolers. As Lewis admitted, “[W]ith rare exceptions the new scouting directors toss all high school players immediately onto the dumping ground.” Oakland has backed away from this attitude since 2002, but it is revealing that the supposedly clear-eyed Moneyballers possessed a clear bias at the time that might have proved as costly to them as more familiar biases, criticized in the book, were for traditionally-oriented teams. Those teams correctly passed on busts like Jeremy Brown for their biases, while Beane’s own bias cost Oakland the chance to draft any of the eight all-stars who signed out of high school in the draft. In other words, it is not clear from Moneyball that Oakland managed to do a better job of identifying the best talent or was much less affected by inaccurate prejudices of its own.

Instead, the great value of Moneyball methods was not better prediction or fewer mistakes, but finding cheaper, though not necessarily better players, and making cheaper mistakes when they occur. Beane’s insight, which Lewis perhaps misidentifies, is not that Moneyball methods find more or better major league players than conventional approaches used by other teams. Each approach finds players that the other misses. And it isn’t clear whether Beane ended up finding more production among players that other teams ruled out, or whether other teams end up finding more production among players that he ruled out. Beane’s insight, instead, was to locate a different set of good players, and miss on a different set of good players, than most of the other teams do, and thereby avoid competing for the same talent and driving up his costs. Although the book emphasizes the supposed superiority of Beane’s methods, it’s the cost-effectiveness of asymmetry that is the more compelling point to take away from Moneyball.

In the end, Oakland’s decline on the field might be understood in part as a combination of two trends. First, as many note, other teams learned to co-opt the Moneyball playbook and negate some advantages Oakland enjoyed early on. But this is only a part of the story, and it is important to note that Oakland has gravitated nearly as much in the direction of traditional approaches since Moneyball’s publication.

Second, Moneyball methods proved somewhat limited over the longer run for Oakland. Moneyball has been insufficient help, at least thus far, for replacing the star players lost over time to free agency. If Moneyball provided a great advantage for finding top talent, Beane should have been able to draft replacements for Barry Zito, Miguel Tejada, Jason Giambi, and other departed all-stars. As Lewis explained, the 2002 draft was “the best chance [Oakland] might ever have to find several Barry Zito’s.” However, as effective as Moneyball proved for finding cheap complementary players like Hatteberg, Moneyball wasn’t as helpful for finding the next set of star players like Zito. Other teams, with their traditional methods, proved very likely to find those star players just as well as Oakland.

Lewis’s thesis of Moneyball superiority assumed the “blunders and the confusions” of other teams, but it turns out other teams aren’t usually so stupid—in fact, Kenny Williams, the White Sox general manager repeatedly mocked in the book, won a World Series only a couple years after the book’s publication. Oakland today still has nice role players discarded by other teams, such as Brad Ziegler and Justin Duchscherer, but it hasn’t been able so far to adapt Moneyball methods to outperform other teams in locating the critical new stars to lead them. Borrowing the words of one baseball scout, Moneyball helped Oakland find “a bunch of safe players,” but “it gets you a losing team” without the star players for them to support. Over the longer run, the greatest beneficiaries of Moneyball seem to be teams, like the Red Sox, that have nicely blended some Moneyball methods with traditional ones, rather than choosing one side or the other in what Lewis depicts too starkly as a baseball culture war.

You may also like...

8 Responses

  1. Larry Rosenthal says:

    I agree; good post!

    Larry Rosenthal
    Chapman University School of Law

  2. In this comment, I express frustration with the conventions of law review discourse that require excessive, stylized, and self-referential but un-self-aware signposting of argument structure. I further argue that whatever the virtues of this style in formal academic writing may be, it is poorly suited to blogging.

    What makes me cry about your opening paragraph is that in the very next sentence — “This is a long, baseball-heavy post, so let me cut to the chase at the outset.” — you switch to a more lively, conversational style. The post immediately comes alive. Embrace the medium. You know you want to.

  3. Linus says:

    I agree whole heartedly with the main arguments here, but I do have a few questions.

    One thing that gets lost in evaluating teams is institutional training. The Royals have become a poster child for the quantz guys to beat up on as they inexplicably seem to focus and train players for skills that seem to highlight their weaknesses rather than their strengths. When Beane and Co. decided to focus on OBP and ignore “tools” so to speak they seemed to take the notion that getting on base was a skill, an easier skill to evaluate and one that was undervalued. But i wonder if one of their innovations was also realizing that their resources being limited, meant less institutional training was available AND less time was available to develop their players.

    I am assuming here that $$ and resources are directly correlated to the quality of training and instruction players can get. A team with less resources goes into a draft knowing two things, 1) they need returns on their drafts faster, 2) they may not get the same result from a super gifted athlete as the Yankess could get. This would naturally mean they would shy away from high school players AND they would be biased for players that already have a skill that can be easily translated to whatever level of ball they play (ie controlling the strike zone).

    I have often wondered whether there the highly touted players who end up as a bust could have succeeded with another organization, or whether there are organizations who are just better at taking less “gifted” players and turning them into stars.

    One limitation with quantitative analysis has always been to make an assumption that development of a player was predictable and equal across organizations. This simply cannot be true.

  4. Linus says:

    Oh sorry. My question is whether or not the new tools in evaluating players, such as F/X would likely be the new wave of innovation. Just reading about the journey of Brian Bannister and how he 1) first tried to use what he learned from statistical analysis for his pitching, but got mixed results, but then 2) started to succeed when he began using his pitch f/x data to better construct a pitching plan that works for him.

    I wonder if the best way to use limited resources for teams is not just to find undervalued skills that can uncover cheaper (not better) players , but develop better training and performance evaluation analysis.

    This post http://www.vivaelbirdos.com/2009/8/19/992832/albert-pujols-anatomy-of-the-swing is fascinating at how it uses Albert Pujol’s swing to challenge traditional notions on how to evaluate swings and hitting in general.

  5. nasamomdele says:

    I think you’ve run too far too fast with the lessons of Moneyball.

    Beane has said that the events of Moneyball were a snapshot of what they were doing at that moment. The lessons have evolved, though some specifics remain- when you don’t have money, you can’t bet on drafting and keeping 1st round picks, so it may be better to use those picks on developable talent. Its just another risky game you play, just with less money.

    To say that the lessons have not played out is both beside the point and wrong. The lessons of Moneyball are divided between the big money teams and the small money teams, like Oakland. You wrote that Boston utilizes some lessons from Moneyball, while they can also afford big money players- what you might call ‘traditional’ approaches to talent valuation. You couldn’t be more wrong.

    In Moneyball, Beane expresses the fact that he would love to land stars for Oakland, but is simply outbid. The lessons are for the competitive benefit of the small-market team, the Davids of sports, not for their dominance, but for their competitiveness.

    Not to mention that the Red Sox are playoff contenders every year with analytical minds running things. Can’t say that for the Yankees.

    Don’t throw the baby out with the bathwater.

  6. That is a great summary of Moneyball. I have always wanted to read that book and this post has made me want to even more. I actually like the idea of Moneyball especially because baseball doesn’t have a salary cap. You see it in every sport teams that draft well and draft smart overcome bad seasons a lot quicker and easier. Also look at how many teams repeat as champions in MLB, there is a new champion every year. I’m interested to see how the movie turns out.