Tagged: empirical legal studies

Any resemblance to current members of the SELS council is purely coincidental.

“Mighty In Their Day:” Reflections on the 9th Annual Empirical Legal Studies Conference

In Tolkein’s legendarium, the 9 rings of power were given to mortal men as a means of their corruption.

“Those who used the Nine Rings became mighty in their day, kings, sorcerers, and warriors of old. They obtained glory and great wealth, yet it turned to their undoing. They had, as it seemed, unending life, yet life became unendurable to them. They could walk, if they would, unseen by all eyes in this world beneath the sun, and they could see things in worlds invisible to mortal men; but too often they beheld only the phantoms and delusions of Sauron. And one by one, sooner or later, according to their native strength and to the good or evil of their wills in the beginning, they fell under the thraldom of the ring that they bore and of the domination of the One which was Sauron’s. And they became forever invisible save to him that wore the Ruling Ring, and they entered into the realm of shadows. The Nazgûl were they, the Ringwraiths, the Úlairi, the Enemy’s most terrible servants; darkness went with them, and they cried with the voices of death. — The Silmarillion, Of the Rings of Power and the Third Age, 346.

The fate of those holding one of the Nine struck me as a useful starting off point for my review of the Ninth Empirical Legal Studies Conference. [For previous installments in my CELS recap series, see CELS III,IV, V, and VI, VII, VIII.1, VIII.2]  The ring-of-power story is apt for several reasons.  ELS is waxing — we’ve obtained “glory and great [relative] wealth,” yet our methods are often described as inscrutable, as we see “things in worlds invisible to mortal men.” One well-known law blogger and sci-fi geek repeatedly has claimed that we behold only Sauron.  Ultimately, there’s a fairly decent argument, based on this year’s conference, that our thraldom — to machine learning — is nigh.  But putting aside the obvious parallels between the world’s leading legal empiricists and Angmar, the witch-king, there’s a far more pressing reason to use the 9 rings as a hook. Multiple sources told me that they found last year’s two-part recap to be “boring” or at best “workmanlike,” asking for more “made-up anecdotes” to spice it up.  So, off we go to Berkeley.

To start, let’s acknowledge the obvious. The West Coast is terribly distant from the home schools of most of the conference’s attendees. (I can’t prove that with data, but I thought this was exactly the kind of unsourced gossip that my readers wanted to see here.)  That was especially true for roughly 200 attendees from the Max Planck institute & the entire faculty of every Israeli law school.  The weather rendered the long trip tolerable, but only just.  Why not bend to reason and just hold a future conference in Germany or in Tel Aviv? Certainly, the conference is now decidedly more international in scope than it was only a few years back.  Was this the result of a maturing discipline, rapidly falling domestic travel budgets, or some unknown missing variable?  Other than the location, which they couldn’t help and probably were proud of, as West Coasters tend to be, the organizers (Anne Joseph O’Connell and Eric Talley) were magnificent and deserve credit for pulling off an enormous project without a hitch.

I arrived in time for both plenaries the first day. In a session on The Future of Big Data and Social Science, I learned that it’s much easier to do social science research when pesky IRBs don’t stand in your way. Though, given recent events, maybe IRBs only get in your way if you bother to tell them you are working on manipulating the political process with your purloined state seals.  In Evidence on Income and Wealth Inequality, Emannuel Saez pitched the utility of very high marginal tax rates as the (only?) solution to persistent and rising inequality. When pushed to articulate whether and how inequality was a social evil, he not surprisingly responded with a market-based argument: i.e., his co-author’s book sales demonstrated the issue’s political salience, and, consequently, the question’s irrelevance.  I thought this was a rather chippy answer, though at the end you have to give it to him.  It may be the least read popular book of the last fifty years, but that’s a ways better than the least read unpopular book!

Read More


The Varying Use of Legal Scholarship by the U.S. Supreme Court across Issues

While patent law is my core area of scholarly interest, I have also studied the use of legal scholarship by the courts. My co-author Lee Petherbridge from Loyola-LA and I have conducted several comprehensive empirical studies using large datasets on the issue. More precisely, we have analyzed how often federal courts cite to law review articles in their decisions. We have empirically analyzed the issue from a variety of angles. We have studied the use of legal scholarship by the U.S. Supreme Court (available here), by the regional U.S. Courts of Appeals (study available here), and by the Federal Circuit (available here). I won’t recount the finding of those studies here. Instead, I will report some new information and ask readers for potential explanations of the data.

Read More


Should Empirical Legal Scholars Have Special Responsibilities?

Before delving into the substance of my first post, I wanted to thank the crew at Concurring Opinions for inviting me to guest blog this month.

Recently, I have been thinking about whether empirical legal scholars have or should have special ethical responsibilities. Why special responsibilities? Two basic reasons. First, nearly all law reviews lack formal peer review. The lack of peer review potentially permits dubious data to be reported without differentiation alongside quality data. Second, empirical legal scholarship has the potential to be extremely influential on policy debates because it provides “data” to substantiate or refute claims. Unfortunately, many consumers of empirical legal scholarship — including other legal scholars, practitioners, judges, the media, and policy makers — are not sophisticated in empirical methods. Even more importantly, subsequent citations of empirical findings by legal scholars rarely take care to explain the study’s qualifications and limitations. Instead, subsequent citations often amplify the “findings” of the empirical study by over-generalizing the results. 

My present concern is about weak data. By weak data, I don’t mean data that is flat out incorrect (such as from widespread coding errors) or that misuses empirical methods (such as when the model’s assumptions are not met). Others previously have discussed issues relating to incorrect data and analysis in empirical legal studies. Rather, I am referring to reporting data that encourages weak or flawed inferences, that is not statistically significant, or that is of extremely limited value and thus may be misused. The precise question I have been considering is under what circumstances one should report weak data, even with an appropriate explanation of the methodology used and its potential limitations. (A different yet related question for another discussion is whether one should report lots of data without informing the reader which data the researcher views as most relevant. This scattershot approach has many of the same concerns as weak data.)

Read More