In Defense of Law Reviews

Criticizing law reviews has been in fashion for quite a while, and in the New York Times there’s a new article with a similar refrain of attacks on law reviews.  In essence, the criticisms boil down to: (1) law reviews should be peer review and articles not selected by law students; (2) many law review articles aren’t cited; (3) practitioners don’t read law review articles.  We’ve heard all these before, and I’m growing very tired of these stale arguments.

Although law reviews are on odd system for publishing, I think that the model is actually not as crazy as it might seem.

1. Is the grass really greener with peer review?

For all their imperfections, students do a fairly decent job. I don’t think that articles in other academic disciplines in the social sciences are any less obscure or are cited more. Peer review is filled with cronyism and with way too much “I don’t like this article because I disagree with it” or “I don’t like this article because I’m not cited enough.”

Although law review editors can get bogged down in silly footnote citation formalities, for the most part, I’ve been pleased with my editing and have received some really excellent editing that has sometimes been more extensive than the editing I’ve received when publishing with academic book publishers.

2. Do we really want to bother with peer review?  Is it still needed in today’s age where there’s no longer a scarcity in publishing opportunities?

Peer review is a “front end” evaluation (prior to publication).  It is designed to determine which scholarship is worthy of publication.  That made sense when there was a scarcity of publishing opportunities.  We wanted good scholarship to be published because being published was something not anyone could do, and it distributed and publicized scholarship.

Today, there isn’t a scarcity of publishing opportunities.  Anyone can publish.  Most articles make it on Westlaw.  Hardly anyone reads the print journals anymore.

Peer review can readily occur on the “back end,” with professors evaluating articles post-publication.

Of course, professors will use law review placement as a proxy rather than read the article and decide its merits for themselves. But this is laziness that professors should blame themselves for. If we want to make things more fair, then professors can be more fair in how they evaluate scholarship and stop using law review placement as a proxy if it isn’t a good proxy.

One reason why professors use law review placement as a proxy is that despite a number of misplaced articles, law review placement isn’t completely random.  It’s not a perfect proxy, but for the most part, the top law reviews publish more articles I that I find to be of quality than lower ranked ones.  Not always, but I don’t need a perfect proxy in today’s age where it is so easy to search for and find scholarship.   It’s a kind of weak proxy that can sometimes be helpful, but it shouldn’t replace making one’s own evaluation.

In the end, if we don’t think law reviews do a good job evaluating scholarship, nothing is stopping us from reading it and deciding for ourselves!

3. Should we be alarmed that so few articles are cited?

As far as articles not being cited that much, most things written and produced are not that good and are forgettable. Think of how many books were written and then think of the great books. The ones that have endured are only a very tiny fraction of the ones published. It takes a ton of rather lackluster stuff to produce diamonds. The process is anything but efficient, but good scholarly ideas aren’t something that can be produced on the assembly line.

Most peer reviewed articles also aren’t that great either.  Historically, very little that was created  — whether it be art, literature, scholarship, etc. — has endured.

4. Should we respond to the gripes of practitioners?

Many practitioners love to dismiss all law review articles, and they can readily point to many that aren’t really written to help practitioners with their research.  There are plenty of pieces of great use to practitioners, and these pieces can be found with not too much effort. But some lawyers and judges think who want to justify their general lack of desire to read any kind of scholarship at all will readily be glib and dismissive and just write off all law review articles.

Some lawyers and judges think that that law professors are supposed to write for them. Since when is it the job of a law professor to do free research for law firm attorneys or judges?  Many law review articles aren’t written for practitioners — they are written for other law professors.  Not all law review articles must be about advocating legal change.  Some articles might elucidate the law and make it more understandable.  They might critique the law.  They might explain judicial behavior.  They might provide a history of the law.  Law professors need not be reform advocates.

Moreover, I have read many a judicial opinion that I know has relied heavily upon a law review article without citing it.  So the statistics about law review articles not being cited doesn’t mean that they lack influence.  They might not be cited because judges just don’t like giving a citation to them.  In many cases, the judge might not have read and relied upon them, but the law clerks might have.

The NYT article also notes how the Supreme Court is citing to fewer law review articles, but this might be because the Court is less creative from a jurisprudential perspective than it used to be.  Many recent Supreme Court opinions have been somewhat workmanlike.  I often have to look back to older opinions to find something interesting going on.

5. Are law review articles too obscure?

The NYT article quotes Chief Justice John G. Roberts Jr. who is an outspoken critic of law reviews.  Chief Justice Roberts said: “Pick up a copy of any law review that you see and the first article is likely to be, you know, the influence of Immanuel Kant on evidentiary approaches in 18th-century Bulgaria, or something, which I’m sure was of great interest to the academic that wrote it, but isn’t of much help to the bar.”

I find the Chief’s comments to be far too dismissive.  . There are many articles irrelevant to the cases he decides, but also many articles of relevance that he doesn’t bother reading as well.  The selection of cases by SCOTUS is hardly immune to the criticism that the topics of most opinions are somewhat obscure, with many of the most pressing issues and ambiguities in law being unresolved by SCOTUS. Before criticizing law professors’ choice of topics and work output, maybe he should look inward at SCOTUS’s.

You may also like...

11 Responses

  1. Speaking as an outsider who sometimes cites to law reviews, I think the peer review problem is more critical in interdisciplinary writing. Law and economics is the obvious candidate, but I’m seeing more and more articles published in law reviews that include statistical work. Very few law students have the necessary background to evaluate the quality of such work.

    For that matter, peer review is no guarantee that articles with methodological flaws or errors in statistical reasoning won’t make it past review, but at least there is a chance that an editor will direct the paper to someone with familiarity with the methods.

    Peer review does have its problems, but I think on the whole the double-blind review process does have some advantages, and certainly avoids the issue of professors “dump[ing] their lesser work onto their own school’s students.”

  2. Marc Roark says:

    Thanks Daniel. Your voice of reason seems quite measured. The one point I’d add is that another under appreciated point of scholarship is it allows professors to refine their voice, which surfaces in the classroom, making the student experience richer and deeper. To this end, not all benefits of the law review system can be measured as “outcomes” such as the author seems to suggest.

  3. brad says:

    The problem with post hoc peer review is that it is difficult for the reader of an article to find. Maybe if there was a big Shepard’s style warning at the top of articles that said “heavily criticized” it would work, but as it stands most bad articles are simply ignored.

    While peer review is hardly perfect (as scandal after scandal shows) at least when you read an article in a well known journal in other fields you can start with a higher prior probability that it isn’t complete nonsense.

    Also, peer review would seem to prevent the crazy system of simultaneous submission everywhere and accompanying expedite process, which is one of the oddest and most tawdry aspects of the law review model.

  4. Daniel Solove says:

    The issue is not whether a peer review system in the abstract is better than the law review system. It is whether the peer review system is worth the time and effort to implement in these days where there isn’t a scarcity of publishing opportunities.

    Quite frankly, I’d rather spend my time reading pieces I know are good and working on my own scholarship than doing a lot of peer reviewing. I can tell rather quickly if a piece is going to be any good, and I can quickly move on if it isn’t. And if I don’t know whether a piece outside my field is any good, I can ask professors in that field whom I trust or see what pieces they recommend on Twitter and in blogs, etc.

    I don’t want to spend tons of time writing detailed reviews of pieces for peer review, as even with a bad piece, considerable time must be spent writing the review. All this is lost time spent with a sub-par piece. Ordinarily, I would quickly brush aside these pieces and move on with more important things, but with peer review, I must write a review. Life is short, and I’d rather not spend my time this way, and I bet many law professors think the same. So we live with an imperfect law review system. We save a lot of wasted time, and . . . yes . . . the system is imperfect, but getting perfection isn’t worth the price.

  5. phil says:

    > Life is short, and I’d rather not spend my time this way, and I bet many law professors think the same.

    That’s kind of a selfish answer. I appreciate the honesty, but still. Peer review in other fields is considered one of the things that responsible academics do. Annoying and boring perhaps, but it helps separate the wheat from the chaff, and multiplied over all the people who have to do their own personal peer review every time they are looking for an article saves net time.

    I see similar attitudes towards grading — particularly good feedback on written work, and towards committee and other institutional work. On top of that, law professors are better compensated as compared to peers in social sciences departments.

    It all makes it very hard to sympathize when law faculty comes under attack in the popular press or in university politics.

  6. Daniel Solove says:


    I think you’re missing my point. Peer review would matter if there were a scarcity of publishing opportunities. But there isn’t. So why bother putting so much time into front end review when there isn’t scarcity anymore and anyone can publish? Why not devote more time to other things?

    The argument that responsible academics just do peer review doesn’t really answer my point about why pre-publication evaluation is important and why post-publication evaluation won’t be sufficient.

    People can readily find the wheat from the chaff by looking at commentary on already-published works. Basically, being published just doesn’t mean much anymore, and I being published never was a particularly precise proxy for being good. It has been a proxy better than random, for sure, but not a precise one, and not one I’d rely heavily upon — even with peer review.

    I think that most debates about peer review seem to be stuck in a bygone mentality where publishing opportunities were scarce. We’re living in a very different world now.

  7. Bob Lawless says:

    As a law professor, I am much more sympathetic toward the “pro peer review” comments. Properly done, peer review is not just a sorting mechanism but instead helps improve the research. My own papers are stronger when they go through peer review, both because of the comments that go with the peer review and because I know when preparing the paper that it will receive more careful scrutiny.

    The biggest challenge for peer-review in law is the normativity of most legal scholarship. Absent agreement on methodology, it is not apparent to me what it means to peer review a normative claim.

  8. Orin Kerr says:

    Dan, I agree with a lot of what you’re saying, but I think you’re overstating the case. There is a scarcity of publishing opportunities in top journals, and we all know that journal placement still matters. To be sure, people *should* read the article and decide for themselves. But a lot of people don’t, and the association with a journal still colors associations with an article. And we see this from decisions scholars make: There’s a reason you decided to publish your recent and forthcoming articles are in the Harvard Law Review, Columbia Law Review, and California Law Review and not secondary journals at low-ranked law reviews.

  9. Adam Levitin says:

    I want to echo Bob Lawless’s point on the problem of peer review of normative scholarship.

    I’d also observe that peer review in small fields just doesn’t work.

    Finally, I’d suggest that these days publication occurs when a work is posted to SSRN, not when it is accepted in or appears in a journal. SSRN posting is literally when the work is made public. The formal publication in a journal is a finishing process (and the inane level of footnoting actually has some virtues), but we’re really in a self-publishing world now. I don’t know anyone who reads actual print law reviews-the SSRN abstracting journals are today’s advance sheets.

  10. Considering that it’s easy and painless to upload a paper to SSRN (or a similar site), the law review serves a function in the sense that a least a paper is screened. I’ve seen some papers accessible on SSRN (I pay for SSRN access … out of my own pocket) that were so absurd — and often biased — that they would never see the light of day in a law review.

    For me, both SSRN rapid publishing and law reviews serve a purpose. Maybe if SSRN was more selective — and sometimes they are — I might opt for SSRN over law reviews. For now, I’ll stick with both.

  11. Larry Rosenthal says:

    I am happy to agree that peer review is far from a perfect system. It may also be true that peer review is difficult to apply to normative legal scholarship — but then again one of the complaints about legal scholarship is that so much of it has a “this I believe” quality. It is for this reason that legal scholarship often tries to ground itself in empirical claims. For just this reason, I think we need to worry that legal scholarship is lacking in adequate quality control because of the lack of rigorous peer review. The process of post-publication review is rather a hit-or-miss thing. It depends on the willingness of other scholars to take the time to scrutinize critically and with rigor the scholarship of others. I wonder how often this happens.

    William Stuntz was one of the most prolific and cited criminal procedure scholars in recent decades, but his work often contained broad empirical generalizations with questionable empirical support — the kind of work that might be questioned by the editors of a peer-reviewed criminology journal, but which law review editors were evidently eager to publish, and legal scholars were sub subsequently eager to cite. For years (really decades), the process of post-publication peer review described in this post seemed to smile on Professor Stuntz’s claims — not because anyone systematically examined the evidence supporting these claims, but because they offered support for others who wished to make arguments that were supported by these same claims. In other disciplines, however, empirical claims are not accepted merely because someone has already made them — they require persuasive supporting evidence. Indeed, criminologists never seemed to take Stuntz especially seriously, but law professors did.

    In their recent reviews of Professor Stuntz’s final book largely recapitulating Stuntz’s law review articles, Stephen Schulhofer and Donald Dripps have dismantled a great many of Stuntz’s central empirical claims. But I wonder whether Stuntz’s work ever would have enjoyed the prominence that it attained if law reviews were more like criminology journals. The more that legal scholars do the work of other disciplines, the less defensible is our system of peer review, or so it seems to me.

    Larry Rosenthal