From the Ivory Tower to the Courts

It has been conventional wisdom for some time that the legal academy has become increasingly disconnected from the practice of law. And because of this, law reviews are said to be of little use to attorneys and judges. A recent piece in the California Lawyer by a law professor and former law school dean has received some attention for making these claims in the strongest terms. From the article:

I did my own count recently of the California Supreme Court opinions published during the past five years that relied on law reviews as authority: There were just six. This despite—or perhaps because of—the fact that law reviews have tripled in number since the 1970s. The 20 ABA-accredited law schools in California now publish a total of 82 law reviews. UC Berkeley’s alone publishes 14, while Stanford and UC Hastings each publish 9. Both law professors seeking tenure and law students seeking employment at elite law firms eagerly fill these volumes. But who reads them now? Surely not the judges who decide the law. And not practicing lawyers either.

As Adam Liptak of the New York Times observed a few years ago, “Articles in law reviews have certainly become more obscure in recent decades. Many law professors seem to think they are under no obligation to say anything useful or to say anything well. They take pride in the theoretical and in working in disciplines other than their own. They seem to think the analysis of actual statutes and court decisions—which is to say the practice of law—is beneath them.”

I’m not quite sure the method that the author used to identify just six citations to law reviews, but I performed a quick and dirty Lexis search and turned up far more. The bigger question, though, is whether the author is right that judges are not reading law reviews. A new article by David Schwartz and Lee Petherbridge indicates that, at least for the federal appellate courts, the conventional wisdom seems to be flat wrong. In fact, according to their study, law review citations have increased dramatically in the last twenty years (even when accounting for the increased number of journals). I couldn’t cut and paste the tables and graphs from the article, but the results regarding the proportion of court opinions that cite law reviews are clear.

So, why is the conventional wisdom so completely wrong on this point? Maybe citations are clerk driven. Or it could be that judges are misremembering the golden age of law reviews. I want to offer a different explanation, though. Judges and commentators have argued that because law review articles are rarely concerned with legal doctrine, they are of no use to those practicing law and judging cases. I think there is a strong argument that it is precisely because law reviews are unconcerned with doctrine, they are much more valuable to judges and are cited as a result. Articles that merely outline, discuss, or analyze doctrinal areas do little to advance the judge’s knowledge beyond what he or she could establish alone. Instead, it is the broader theoretical points and empirical studies that are outside of a judge’s metaphorical wheelhouse. Just as judges will not allow an expert on law to testify, it makes little sense for law reviews to inform judges about topics which they are comfortable. Instead, like the expert on ballistics evidence or biological sciences, law reviews that are increasingly removed from “law” actually educate judges where they are weakest. And, as a result, law reviews are increasingly being cited by the very judges who proclaim their uselessness.

You may also like...

13 Responses

  1. A.J. Sutter says:

    1. Prof. Uelmen’s count is 6 distinct Cal Supreme Court opinions published in past 5 years, relying on (i.e., not citing in dicta ) law review articles — not six citations, as you mention of your search. Just wondering, did your quick-and dirty search conform to Prof. Uelmen’s criteria?

    2. Your criterion of usefulness seems to apply to litigation only (and indeed, only to cases that make it to an adjudicated appeal). As a transactional practitioner, I can say that I can’t recall any article from a student-edited law review that has helped me, despite my continually seeking for them, in a quarter-century of practice. (Out of an excess of caution, I may have estimated this as “one or two” in prior comments to other posts on this blog in the last couple of years.)

    3. There are also questions raised by, and questions not asked in, the Schwartz & Petherbridge article that should temper your view that “the conventional wisdom so completely wrong on this point.”
    (a) S&P don’t mention the absolute number of law review articles cited — maybe a small number get a lot of cites, just as with the statistics of websites.
    (b))They also don’t mention the proportion of published articles cited. Nor do they mention the age distribution of articles cited. (They assume that Justice Scalia is right when he says the shelf life of a law review article is 5 years, but why should they accept this anecdotal judgment when they “prove” that Justice Roberts’s anecdotal estimation is unreliable?) As Prof. Uelmen pointed out, the number of law journals has exploded in the past 20 years or so. So even assuming for the sake of argument that the absolute number of articles cited has gone up, the relative number of “useful” articles may have decreased. That alone could explain why most judges and practitioners complain.
    (c) S&P mention that a relatively small cohort of judges (< 14%) are responsible for ~ 50% of all citations. They don't mention who these judges are — might they be concentrated in particular circuits?
    (d) S&P also don't discuss the content of the articles cited, or the types of cases that tend to cite them. E.g., patent cases might correlate with the Federal Circuit, which didn't exist during the first half of the period discussed by S&P. Or article content might reflect a particular ideological shift, e.g. more weight to a law & economics approach, which might correlate with particular judges as well as explain, due to the temporal dynamics of ideological reception, a recent up-trend in cites.
    (e) Especially in light of the possibility of an ideological correlation, one also needs to ask: if it's true that law review articles are having more influence on judges despite being more oriented to theory and "law &…," is that unequivocally a good thing for society (as distinguished from being good for law profs)? What is the substantive content of what's being cited?

  2. lawanon says:

    Your hypothesis re judges citing articles that are *not* doctrinal analysis is easily testable, and I wish you attempted to test it before posting it. Look at the six mentioned CA supreme court opinions and see whether law review references are related to doctrinal analysis or non-doctrinal research. I’d bet it’s the former, though we’ll never know until someone does the reading and reports here.

  3. TJ says:

    One point that everyone seems to miss is that there is no shortage of doctrinal scholarship being published in law reviews. They are called student notes. Lawyers and judges who complain that law reviews are useless because they don’t contain doctrinal analysis thus (1) overestimate how useful they actually find doctrinal scholarship, (2) are ignorant about what is actually in a law review, or (3) have a rather elitist (is there a better word here?) preoccupation with the prestige of the author rather than the merit of the content.

  4. Law professors do have something important to contribute even when they “merely outline, discuss, or analyze doctrinal areas.” We have time: time to read and time to think. Even a highly conscientious judge will receive an issue as it has been framed by the parties, and is under institutional pressure not to go off on research fishing expedition. A professor who takes the time to study an issue thoroughly can bring back fresh insights, unexpected precedents, compelling syntheses, and surprising challenges to seemingly settled authority. This kind of work can be done well or done poorly, but done well, it is institutionally valuable.

    Also, TJ’s point about student notes is excellent. Advisors: steer your students away from common topics. A student who writes about whether the DMCA is stifling creativity will say nothing new and will never be cited. A student who notices that Pennsylvania courts apply two startlingly inconsistent choice-of-law tests in insurance cases will say something interesting and may well have it picked up by the courts.

  5. Corey Rayburn Yung says:

    Hi A.J.,

    I am not sure I can properly distinguish the difference between “citation of” and “relying on” and I do not know how Uelman made such a distinction (if he did). I’m dubious of your dicta differentiation because I can’t imagine an instance where a law review is actually part of the holding. It seems like such a citation is always dicta (unless the facts of the case actually pertain to a law review which would be irrelevant to this discussion).

    As for usefulness, I agree that nothing I said pertains to transactional lawyers. Since I was primarily focused on judges and their collective claims of the irrelevance of law reviews, I only meant to speak to litigation.

    As to your methodological objections, you would have to speak directly to the authors. However, I wouldn’t expect them to present the data with every breakdown possible. And since some judges and commentators have argued that judges have no use for law reviews, I feel that they have effectively attacked that claim.

    Hi anon,

    Since this was not my study, I did not attempt to determine if the doctrinal citations are more or less common. I would disagree that it would be “easy” to do since it would require a substantial sample size and a lot of labor to categorize the law reviews cited. Perhaps Schwartz and Petherbridge will perform such an analysis, but as the mere author of a blog post, I didn’t want to start such a large scale project.

    Hi James,

    I did not mean to discount the value of doctrinal work. I actually wrote a purely doctrinal article about the Adam Walsh Act and it has been cited by a couple of federal courts perhaps for the reasons you cite. I only meant to point out that the shift away from doctrine might end up being more helpful to judges insofar as cases involve areas of research in which judges are not naturally experts.


  6. Mark McKenna says:

    I have always been unpersuaded by the claim that legal scholarship is unimportant because it is not cited by judges (and that was even assuming, perhaps contrary to the evidence, that it was in fact true that judges were not citing law review articles). Citations in decisions are simply not a good proxy for influence of legal scholarship. First of all, our scholarship affects our teaching, and the students are exposed to the ideas in the scholarship whether they know it or not. Trends in the scholarship therefore shape new generations of lawyers. I doubt lots of judges were citing articles written by the legal realists, but certainly they were having an important impact on the law. (I suspect something similar could be said about the originalist movement). Second, in many areas of law, including my own, professors regularly write amicus briefs and/or assist practicing lawyers. Whether they are cited directly or not, the arguments in academic articles are presented to courts and shape the form of the arguments. So I just don’t see why the citation count is important.

  7. Dave Schwartz says:

    As a co-author of the study, I wanted to chime in. First, I agree with Corey Yung’s responses to the comments. Second, I also agree with A.J. Sutton and Mark McKenna that there are additional uses of scholarship beyond citation in opinions. Our paper makes this exact point. There is also a substantial body of literature relying upon citation counts. Our paper engages the citation count literature. I believe we have provided new information to the debate, and analyzed citation data in new ways from what has been done in the past. Many of A.J.’s questions — such as what percentage of published articles are cited and what are the subject matters of the opinions citing legal scholarship — are merely possible directions for future research. In fact, we flagged several of these areas in our paper. His inquiry about the small cohort of judges who cite scholarship is easily answered. (A.J.: “S&P mention that a relatively small cohort of judges (< 14%) are responsible for ~ 50% of all citations. They don't mention who these judges are — might they be concentrated in particular circuits?"). The 10 judges who cited to scholarship in the most opinions from 1990-2008 reside in 7 different circuits, and no circuit is home to more than 2 of them.

  8. A.J. Sutter says:

    Thanks, Corey & Dave, for your replies. My objections were less to methodology than to the inferences Corey’s post made from the results.

    The study mentions that about 3.4% of opinions cited legal scholarship in 1950-1979, and about 4.8% cited it from 1980-2008. First of all, that’s not a whole lot, in absolute terms. Neither is the the 6.21% during the shorter period of 1999-2008 — it means that > 93% of the time, the appellate courts didn’t cite scholarship. If my clients told me that 93% of my work was useless, I’d rightly feel like a failure. Of course, the same would be true if they felt that way about a considerably smaller percentage (just to put your “other uses” argument into perspective). You might reasonably argue that not all of that 93% is intended to help lawyers and judges anyway, but that makes their complaint more plausible, not less so.

    In relative terms, it’s about a 40% increase in citation between the two periods. Here’s where the issues raised in my items (a) and (b) are pertinent. Let’s first assume that the cites are evenly distributed among published articles (i.e., probability of citation is the same for each published article). It’s very difficult to establish this probability density a priori (and it isn’t very plausible anyway — otherwise there wouldn’t be “top tier” journals, etc.), but it’s a simplifying assumption that provides an easily-tested case: if the average annual rate of publication during the later period increased more than 40% compared to the rate during the earlier one, the proportion of useless articles will have increased. (Since my own alma mater has > 2x as many law journals as it did when I attended in the early 1980s, I find such a large increase in the rate of publication prima facie plausible.) If instead the citation of articles isn’t evenly distributed, then it could be a much more complicated calculation, but one possible result is that an increase in the rate of publication of < 40% can nonetheless lead to an increase in the proportion of useless articles.

    A greater proportion of useless articles would certainly justify the complaints of lawyers and judges, and yet could be entirely consistent with S&P's published data. So in fact the S&P study is inconclusive as to this controversy.

    BTW, I’m not so sure it’s reasonable to be so blasé about the uselessness of law review articles to transactional attorneys. Law school isn’t solely for the training of litigators and judges — Mark McKenna’s comment that your scholarship affects your teaching is pertinent on this score. And in fact a lot of the subject matter taught in law school is predicated on the existence of transactions.

  9. Matt says:

    AJ- surely you don’t think there’s a valid inference from “not cited by a judge in an opinion” to “not useful for a judge in understanding an area of law.” (And, of course, something can be useful for law students and other lawyers without being useful for a judge, too.) That’s a complete non-sequitur. I don’t, of course, make the converse claim that all (or even most) legal scholarship is _useful_ for judges, but the inference you make here is completely unwarranted, as I’m sure you’ll see.

  10. A.J. Sutter says:

    Slight correction: strike my comment from “If my clients told me that 93% of my work was useless, …,” since mathematically this is too simplistic. There isn’t any necessary connection between the percentage of appellate cases citing legal scholarship, and the percentage of sources of such scholarship that are cited. E.g., even if 100% of appellate cases in a given year cited legal scholarship, that wouldn’t necessarily mean that 100% of legal scholarship was cited; and even if 100% of legal scholarship were cited, it wouldn’t necessarily mean all cases included at least one citation.

  11. A.J. Sutter says:

    Sc. strike from that phrase to the end of that paragraph only. Apologies for the itchy mouse fingers.

  12. A.J. Sutter says:

    Matt, the issue in my earlier comments was Corey’s inferences from the case cite data in the S&P article. As for your point, we don’t have any way of knowing whether an article is influential or not in the way you propose. Law professors and students can think of ways in which their work might have influence invisibly, and I’m not claiming they (you) are categorically wrong. However, I’m not sure why this merits more weight than the testimony of judges and lawyers themselves about what they find useful vel non.

    I also don’t think most practitioners and judges are arguing the absolute, that law review articles are never useful, including in the diffuse way you suggest — it seems to me that the point is a more relative one, like there being so much stuff to wade through, and so much of it not informed by practical experience or concerns, that it’s getting noticeably harder to find something useful than it used to be. It’s not my wont to put things in such terms, but perhaps one could say that for practitioners the transaction costs of dealing with legal scholarship are higher than they used to be.

    Or another economic analogy: for law scholars to “prove” their own usefulness through statistics (not that S&P have done so) is a bit like economists saying householders are “better off” because per capita GDP goes up. If householders nonetheless report feeling worse off, that’s something worth listening to.

  13. Corey Rayburn Yung says:

    Hi A.J.,

    I did not intend for my comments to be construed as an inference from the study. I was arguing in favor of one of many possible working theories. Further research would be needed to test my hypothesis.

    And I certainly do not believe that court citation is near the most significant factor in assessing the importance of scholarship. I just find it interesting that judges have been arguing that law reviews are increasingly irrelevant while their actions seem to tell a different story. I think the distance between the ivory tower and the courts is smaller than most believe and the S&P study supports that conclusion.

    As for the importance to transactional lawyers, I have no comment. Since my primary fields are criminal law and federal courts, I’m almost exclusively focused on litigation issues.