Richard Lempert, guest-blogging at the ELS Blog, has a great series of posts on empirical scholarship in law. In the first, he observed that:
Too often researchers encourage misuses of their results in conclusions that push the practical implications of their research, even when the more detailed analysis emphasizes proper cautions. While this occurs with empirical students of the law in liberal arts schools by political scientists, sociologists, economists and psychologists among others, the problem tends to be more severe in the empirical work of law professors, perhaps because most see their business not as building social or behavioral theory but as criticizing laws and legal institutions and recommending reform.
In the second, he said:
There is also the question of qualitative data. I am distant enough from the ELS movement that I do not know how its core advocates regard qualitative research, but taking down 5 volumes of the Journal of Empirical Legal Studies that happen to be close at hand I could not help but note that every article in every volume had a quantitative dimension. Each had at least a graph, table, equation or regression and most analyzed and presented results using more than one of these analytic modality. Yet qualitative research is as empirically-based as quantitative research and it can be as unbiased and as rigorous. Moreover, it is often more revealing of relationships legal scholars seek to understand, not to mention more accessible and interesting. Lawyers have done many quantitative studies I find useful and admire, but I would not elevate any of them above, for example, Bob Ellickson’s study of Shasta county when it comes to developing and sharing an understanding of the real world or, in this case, illuminating the limitations of the Coase Theorem.
And most recently, he argued for a deeper appreciation of the role of ground-tested theory:
What is plausible depends, of course, on what we know about the matter we are studying. More than occasionally empirical scholars seem to have little appreciation of context beyond the general knowledge everyone has and the specific data they have collected. Without a deep appreciation of context, even the best scholars may be misled. For example, some years ago Al Blumstein and Daniel Nagin, who were and are among the very best of our nation’s quantitative criminologists, did a study of the deterrent effects of likely sentences for draft evasion on draft evasion rates. For its time the study was in many ways exemplary – variables were carefully measured and analyzed, and it was refreshing to see an investigation into deterrence outside the street crimes and capital punishment contexts. The results of the Blumstein-Nagin research strongly confirmed deterrence theory. Resisting the draft by refusing induction was substantially higher in those jurisdictions that sentenced resisters most leniently. Yet I regarded the study as worthless.
To find out why, and to read more of this powerful (but friendly) critique of the newly dominant methodology in legal scholarship, check out the ELS blog!