Outside Reviewers Stay Anonymous!

Outside reviews of manuscripts are important to many publishing enterprises, from scholarly books and articles to general interest works of nonfiction.  Honest, objective and impersonal assessments are vital. 

That is best promoted when the identity of the reviewers is held strictly confidential, by editors and reviewers alike, with editors sharing only the substance of reviews to enable improving a manuscript. 

Contrary to this normative ideal, reviewers often seem to feel free to identify themselves, and even editors are sometimes sloppy in leaking identifiying data.   In just the past year, I have personally had several different unfortunate examples.  The upshot is the same: outside reviewers must stay anonymous.

In one case, I was asked to review a scholarly manuscript of an article on a subject within the field involving law and accounting.  I gave a generally favorable review, along with some quibbles and suggestions.  The editors tentatively declined the piece with an invitation to resubmit based on the quibbles and suggestions. The author somehow figured out that I was the reviewer and contacted me to say so.  It felt as if I were being accused of interfering with some one’s scholarly work.  Not pleasant.

In another case, I submitted a book proposal to a university press. The press expressed great enthusiasm and the editor told me it had to follow standard practice and have outside reviews.  It circulated the proposal, including sample chapters to several anonymous reviewers.  One of them, an old acquaintance of mine, told me he was a reviewer and that he really liked one of the chapters. 

Again, I wish he hadn’t told me this but, having told me, I followed up by asking his opinion and seeking his comments on the book.  Bizarrely, he refused to discuss any aspect of his review or the book.    Worse, the interest from the publisher went totally cold. 

The connection was hard to miss.  It seems clear the acquaintance both wanted to signal his power to interfere without providing the substantive assessment which, oddly, the press never did either.  All in all, it would have been better for this fellow to keep himself anonymous.

In a third case, a different university press, also very enthusiastic about my book proposal, supplied me with all the outside reviews it commissioned, and offered to publish the book.   In the meantime, one of the reviewers, while communicating on a different subject, mentioned that he served as such. I said I appreciated the review.  

What I didn’t say, but might have, was that I wish I had not known that he was the reviewer. The knowledge obviously did not produce the negative feelings I got when from the other two episodes. But I still would rather not know.  

The norm of anonymity for outside reviewers exists for good reasons.   It should be respected.

You may also like...

11 Responses

  1. Orin Kerr says:

    I see this as a bit more complicated, Larry. Often that’s true, but I think there are some instances in which it’s appropriate (or at least not a bad thing) to volunteer identity. At the very least, I’m not so sure of the harms.

    Let’s say a publisher contacts me about a book proposal and asks for my review. They want me to take a a lot of time to read the book proposal or draft of the book and then write up a sustained series of comments on the merits of the book. Unlike a tenure letter, the review is not for an academic audience: It is not going to go in anyone’s file, or be discussed among colleagues as helpful to the workings of an institution considering a tenure case. The review is really just for the benefit of some private company that wants me to help them. Maybe they’re going to pay some small amount that amounts to about minimum wage to write the review, but they might just do it for free. If that’s the case, why would I bother?

    One of the few reasons a person might have for accepting the invitation to write the review is to be able to contact the author at some point in the future and — if the review was positive — let them know that you helped them get the book contract. It’s a very small benefit, of course: You might not like the book, and then you’ll probably stay mum. But if you liked the book and helped the author, it may be a small incentive to take the time to write the review to help them and then let them know about it.

  2. Steve says:

    Interesting… thank you for posting this!

    Steve
    Common Cents
    http://www.commoncts.blogspot.com

  3. A.J. Sutter says:

    Orin, the downside of what you suggest is that the author might feel uncomfortably in your debt. It sounds as if that was Lawrence’s third case. It’s for this reason that giving anonymously to a known recipient ranks as high as level 3 in Maimonides’s 8 levels of charity, whereas giving it adequately after being asked is only level 6.

  4. TJ says:

    Orin, the problem with your suggested regime is that it creates asymmetrical incentives. Reviewers know that they will get mutual-backscratching credit if they write a positive review, and get nothing if they write a negative review. So everyone will end up writing a positive review. That defeats the point of the process.

  5. Isn’t the reason for accepting the review supposed to be that one wants to participate in the advancement of knowledge in one’s field? Given that role and the economics of academic publishing, I don’t see such a sharp distinction between an academic press and a tenure committee as audiences for one’s pro bono analysis of a work.

  6. Orin Kerr says:

    TJ,

    I agree that’s a problem, as every publisher knows. At the same time, it’s cheaper and more effective for the publisher to discount for that possibility than it is to have no reviews at all or else pay a serious fee for the review: I gather that’s why the publishers do just that.

    AJ,

    I agree. I’m not saying that the system I am suggesting is perfect, just that it is not necessarily the worst of the realistic options.

  7. Orin Kerr says:

    Jennifer,

    I agree that this is one possible reason to accept the invitation, and it is the ideal one. But the question is how often that ideal motive is actually driving the decisions. My sense is that it is only rarely.

  8. TJ says:

    But I’m not sure that “publishers do just that.” Sure, they don’t contractually bar reviewers from disclosing that they authored a review. But there is an existing scholarly norm against such disclosure, and there certainly is a norm against biasing your review in the hope of getting reciprocal benefits in the future, upon which the publishers may well be relying. It seems to me that the point of Larry’s post is that the norm is being weakened. And you cannot sustain the anti-backscratching norm in the long run without the anti-disclosure norm.

  9. Orin Kerr says:

    TJ,

    I think we just disagree about prevailing norms: It seems to me that there is a major gap in this area between what we say we *should* do and what we actually do.

  10. frankcross says:

    Ideally, they would not be anonymous, which destroys accountability. The theory is that they won’t be sufficiently critical if they risk antagonizing the author. In this case, the point is a good one — disclosure risks reviewers sucking up to an author rather than speaking his or her mind. And I think Orin is a little too cynical here — I believe most reviewers do it for the purpose of participating in scholarship — while simultaneously insufficiently cynical — the incentive he describes is one that would call for unjustified praise in order to receive it.

    But if you really know the area, there’s a decent chance you can guess your reviewers by the likely candidates and the content of the review.

  11. TJ says:

    Orin,

    I’m not sure we disagree about the norm. I think we agree on what we say we *should* do, and we agree that people don’t often live up to it. What we seem to disagree is what to do about the gap. You seem to think that the gap means that the aspiration is either futile at best or sadly misguided at worst. I think the aspiration is normatively desirable — perhaps impossible to maintain and thus futile, but normatively desirable nonetheless.