Everything Is Empirical

You may also like...

8 Responses

  1. Joe says:

    This is very apropos of a current project. I’m looking at an area of law that has evolved very differently in federal courts vs. state courts (which are nearly uniform on this issue). A few scholars have offered abstract theories to explain the difference in recent years, but no one (that I can find, at least) has ever tested these theories to see if they are, well, right. So I am going back through all fifty states and tracing the evolution of the law in this particular field to see if confirms or refutes the theories. (The results so far are unsurprisingly mixed.)

    Is this “empirical” work in a broad sense? Of course. I’m using real world observations to evaluate an abstract idea. But it’s certainly not an ELS piece; there will be very few numbers at the end of the day, and my results are likely to be of the squishy type in a lot of cases (e.g., the law in State X developed sorta kinda in a way that the theories would predict, but not exclusively so).

  2. A.J. Sutter says:

    Good insight, and no need to be so tentative or apologetic about suggesting why the distinction is important. If anything, you should be more assertive.

    Very likely the equation of statistical (usually econometric) methods with “empirical” ones grows out of the law & economics trend, which appropriates neoclassical economics’ own math-envy of the (’empirical’) natural sciences. My impression is that qualitative researchers in law seem to prefer to emphasize that they’re using, say, an anthropological approach or a sociological one, rather than an “empirical” one (am I mistaken?) — that too may arise from the cultural habits of sociologists and anthropologists, and their attention to differentiating one field from the other.

    Your footnote is especially rich. The ELS usage (“Its’s empirical“) calls to mind the old Duck’s Breath Mystery Theater character Mr. Science, who used to trump all doubts with his boast “I have a Master’s degree — in Science!”

  3. Bob Lawless says:

    David, I enjoyed your post and have never been too happy with the label “empirical legal studies” for many of the reasons you describe.

    I don’t think you would get much disagreement from most anyone who does heavily quantitative empirical work. Of course, good qualitative empirical work is valuable and often can succeed on a topic where quantitative work cannot. Whether it is qualitative or quantitative, good empirical work is systematic, replicable, and valid.

    Where I think you get pushback is the use of the term “qualitative empirical research” as cover for “bad empirical research.” I have heard faculty workshops where someone presents a few stray interviews as “qualitative empirical research,” when they should have said, “I called up a few people I know, and they agree with me.”

  4. In July of last year there was a blog post by John Inazu (with a comment by yours truly) at The Faculty Lounge that raised similar issues: http://www.thefacultylounge.org/2010/07/some-reflections-on-empirical-legal-studies.html

  5. dave hoffman says:

    Hi Dave
    This is a nice post – though I agree with Bob that good qualitative work is of course empirical and considered so by people in the field. Though I believe that CELS has a bit of a quantitative bias , which is unfortunate.

    I also agree that there is really a ton of bad “I sent out surveys” or “I talked with lawyers” work out there. Whether it is “empirical” or not seems somewhat besides the point.

  6. A.J. Sutter says:

    Bob, qualitative research (as such, and not as “qualitative empirical research”) has a longer pedigree in anthropology, sociology and other social sciences. While it’s hard to call anything rigorous in a field that mixes positive and normative aspects, there is a good deal of learning from a history of practices — try an Amazon search on “qualitative research”. Statistical research is well-known for being capable of abuse, too, and suffers from the further drawback that, both because it is more arcane and because of our undue respect for the quantitative and for the “average” case, it’s easier to bamboozle people with statistics than with qualitative research.

  7. David Fagundes says:

    Thanks, all, for the interesting comments. I think there are a few propositions that we can agree on. The first, totally uncontroversial proposition, is that good scholarship is good scholarship, regardless of nomenclature. The second proposition, slightly less obvious but still apparently unobjectionable, is that qualitative research merits the moniker “empirical” as much as quantitative research does. Lurking in there may be a third proposition, that work’s status as “empirical” or otherwise is orthogonal to its quality.

    So is there anything controversial here? Possibly. There may be some disagreement as to whether it matters that some empirical work is deemed “empirical” while other empirical work is not. One could say (and I think this post by Michael Heise over at the ELS Blog suggests: http://www.elsblog.org/the_empirical_legal_studi/2011/08/more-musings-on-what-is-empirical.html) that calling work “empirical” is value-neutral, so that there are no positive connotations conferred by the term. On this view, “empirical” is just an innocuous descriptive.

    In a vacuum, I think that’s probably right. But the concern I have is that referring only to quantitative work as “empirical” creates a negative implication that qualitative work is _not_ empirical. This suggests that the latter lacks the same degree of rigor and precision as quantitative methodologies, and may cause it to be taken less seriously or regarded more poorly by those in or out of the field.

    Of course, it may be the case, as Bob and Dave have pointed out, that qualitative empirical work could actually lack rigor and precision, as where a few phone calls are held out as sufficient to prove some factual proposition. But I don’t think qualitative work is distinctively susceptible to this charge. AJ correctly points out that quantitative work can be executed poorly as well, and that there may be a particular concern when that happens. Most people can find the error when someone makes a few phone calls and presents that as the basis for “empirical” research. But even sloppy errors in quantitative work are hard to spot (at least for those of us without formal training).

  8. A.J. Sutter says:

    This thread provides evidence of a sort of paradox. On the one hand, legal scholars seem to be reluctant to recognize scholarship in other disciplines. Take the fixation on the red herring of “a few phone calls” as qualitative research: even if some conference presenters did make this silly error, some comments seem to find it prima facie plausible that maybe that’s more or less all there really is to doing qualitative research (even if some comments try to defend the research from the charge). Any cultural anthropologist would find this view about as plausible as we might find the Tea Party’s statements about the federal debt ceiling, or Sarah Palin’s pronouncements on foreign affairs. I.e., a little interdisciplinary exposure is all it takes to recognize the charge as obviously silly, and not meriting such earnest angst. If the lousy qualitative research methods are in fact more widespread in the legal academy, as Dave H. suggests, than just a few silly papers, this just proves my point about the parochial POV. That parochiality would be a better object for the angst.

    On the other hand, legal scholars (among others) nonetheless crave to borrow the authority of other academic disciplines. A big reason for using the word “empirical” is to borrow the authority of the so-called hard sciences (from which the notion of “value-neutral” is also borrowed). That borrowed authority is also the source of any negative implication of being non-empirical, which David F. worries about. Such a negative implication per se may be less a cause for concern than is the power-grabbing pretense that gives rise to it.

    A compromise might be to re-identify ELS as “Econometric Legal Studies”. That way the acronym could stay intact, and the methodology would be more explicit. Resistance to this could be a good diagnostic that the authority grab is the real motivator in the movement.