Author: Dave Hoffman

0

Introducing Guest Blogger Stephen Galoob

I’m pleased to introduce guest blogger Stephen Galoob, who lasted visited with us in 2013. Stephen is an assistant professor at the University of Tulsa College of Law. He is a graduate of UVA law school and received his Ph.D. at U.C. Berkeley’s Jurisprudence and Social Policy program.  Stephen teaches criminal law, criminal procedure, legal ethics, and an undergraduate course on moral and legal responsibility.

Stephen’s recent work includes:

Norms, Attitudes, and Compliance, 50 Tulsa Law Review 613 (2015) (with Adam Hill)

Intentions, Compliance, and Fiduciary Obligations, 20 Legal Theory 106 (2014) (with Ethan Leib)

Are Legal Ethics Ethical? A Survey Experiment, 26 Geo. J. Legal Ethics 481 (2013) (with Su Li).

1

The Civilizing Effect of Legal Training

The cultural cognition project has a new article out on how motivated cognition interacts with professionalism:

This paper reports the results of a study on whether political predispositions influence judicial decisionmaking. The study was designed to overcome the two principal limitations on existing empirical studies that purport to find such an influence: the use of nonexperimental methods to assess the decisions of actual judges; and the failure to use actual judges in ideologically-biased-reasoning experiments. The study involved a sample of sitting judges (n = 253), who, like members of a general public sample (n = 800), were culturally polarized on climate change, marijuana legalization and other contested issues. When the study subjects were assigned to analyze statutory interpretation problems, however, only the responses of the general-public subjects and not those of the judges varied in patterns that reflected the subjects’ cultural values. The responses of a sample of lawyers (n = 217) were also uninfluenced by their cultural values; the responses of a sample of law students (n = 284), in contrast, displayed a level of cultural bias only modestly less pronounced than that observed in the general-public sample. Among the competing hypotheses tested in the study, the results most supported the position that professional judgment imparted by legal training and experience confers resistance to identity-protective cognition — a dynamic associated with politically biased information processing generally — but only for decisions that involve legal reasoning. The scholarly and practical implications of the findings are discussed.

Kahan and I have gone back and forth about how best to characterize the results of the study. He, modestly, seeks to constrain the inferences to the data and to a push back against the vulgar understanding of the judiciary as merely housing politicians in robes.  I think the study speaks to something larger still — the value of legal education & experience in producing situation sense, which enables lawyers and judges (and, to a lesser extent, law students) to agree on the results of legal outcomes notwithstanding their political and ideological priors. Such legal judgment is, after all, one of the practical skills that law school conveys, and which it ought to boast about.

1

The Significant Decline in Null Hypothesis Significance Testing?

The likelihood is higher that this portrait represents Bayes than it does me.

The likelihood is higher that this portrait represents Bayes than it does me.

(Cross-posted at Prawfs.)

Prompted by Dan Kahan, I’ve been thinking a great deal about whether null hypothesis significance testing (NHST, marked by p values) is a misleading approach to many empirical problems.  The basic argument against p-values (and in favor of robust descriptive statistics, including effect sizes and/or   Bayesian data analysis) is fairly intuitive, and can be found here and here and here and here.  In a working paper on situation sense, judging, and motivated cognition, Dan, I, and other co-authors explain a competing Bayesian approach:

In Bayesian hypothesis testing . . .  the probability of obtaining the the effect observed in the experiment is calculated for two or more competing hypotheses. The relative magnitude of those probabilities is the equivalent of a Bayesian “likelihood ratio.” For example, one might say that it would be 5—or 500 or 0.2 or 0.002, etc.—times as likely that one would observe the results generated by the experiment if one hypothesis is true than if a rival one actually one is.

Under Bayes’ Theorem, the likelihood ratio is not the “probability” of a hypothesis being true but rather he factor by which one should update one’s prior assessment of the probability of the truth of a hypothesis or proposition. In an experimental stetting, it can be treated as an index of the weight with which the evidence supports one hypotheses in relation to the another.

Under Bayes’ Theorem, the strength of new evidence (the likelihood ratio) is, of course, analytically independent of one’s prior assessment of the probability of the hypothesis in question. Because neither the validity nor the weight of our study results depends on holding any particular prior about the [question of interest] we report only the indicated likelihood ratios and leave it to readers to adjust their own beliefs accordingly.

To be frank, I’ve been resisting Dan’s hectoring entreaties arguments to abandon NHST.  One obvious reason is fear: I understand the virtues and vices of significance testing well.  It has provided me a convenient heuristic to know when I’ve “finished” the experimental part of my research, and am ready to write the over-promising introduction and under-delivering normative sections of the paper.  Moreover, p-values are widely used by courts (as Jason Bent is exploring).  Or to put it differently, I’m well aware that the least positive thing one can say about a legal argument is that it is novel.  Who wants to jump first into deep(er) waters?

At this year’s CELS, I didn’t see a single paper without p-values. So even if NHST is in decline, the barbarians are far from the capital.  But, given what’s happening in cognate disciplines, it might be time for law professors to get comfortable with a new way of evaluating empirical work.

1

Proceduralists’ Shibboleths

Would Charles Clark really care about plausibility pleading?

Would Charles Clark really care about plausibility pleading?

(Cross-posted at Prawfs).

Recently a call for nominations came out on the civil procedure listserv: what’s the worst civil procedure case ever.  Nominations poured in–even as Pepperdine’sexcellent symposium on this worst topic was all-but-ignored. Sadly, recency bias trumped careful thought, and a plurality of respondents focused on Twiqbal.  In some ways this is an unsurprising result. Twiqbal hit a sweet spot for modern scholars.  The decisions together appear to be politically conservative (fitting modern progressives’ newfound suspicion of the Supreme Court); they cry out for empirical examination (fitting modern scholars’ newfound love of counting things); and they produce a test whose indeterminacy makes socratic dissection easy.

But here’s the thing: dozens of scholars have spent enormous effort on these problems, and have found essentially no observable effects on party and judge behavior, whether in  or out of Court. In that way,  Twiqbal is a black hole for scholarship — its sucks in quants and non-quants alike in, but nothing comes out.

Consider two recent papers — one by Jonah Gelbach, forthcoming in Stanford, and one by Roger Michalski and Abby Wood, under review.  As a part of a dazzling empirical & game-theoretic analysis, Gelbach points out that “a reasonable observer could conclude that the heated debates over the empirical evidence on Rule 12(b)(6) motion grant rates haven’t—couldn’t—shed any light at all on the actual effects of Twombly and Iqbal.”  (Emphasis added.) Michalski and Wood, studying state adoption of Twiqbal,  conclude that whether “at the federal or state level, attorneys and judges are either not as attuned to procedural changes as many commentators think they are, or plaintiffs were already pleading with factual specificity so as to negotiate earlier and more favorable settlements.” And yet, as they point out, “many academics, practitioners, and commentators simply refuse to believe that the switch from notice pleading to plausibility pleading would not have an empirical effect.”

What’s going on? Is this motivated cognition by progressive proceduralists, who can’t admit that the worst cases of their generation (or any!)  had no measurable effects? (That’s not to say that Twiqbal hasn’t had an effect in the world – just not one that is observable.)  Because their priors are so strong, later evidence is discounted.  As such, Twiqbal is quickly becoming a progressive proceduralist’s shibboleth: to belong to the academy community (and to be welcome at conferences), one has to agree that plausible pleading is implausible, evil, and otherwise wrongheaded.  Defending the decision is like defending Lochner. It can be done, but you really ought to teach at Mason.

Or is it something else? Maybe Twiqbal has attracted attention not because it actually represents a change in practice today (after all, no one was truly engaging in notice pleading) but rather because the cases represent a watershed in procedure – the beginning of a return to a pre-1938 code or fact pleading regime. Like Doleor Printz, it’s a signal of a revolution that’s coming. My colleague Craig Green has worked over the last several years to identify certain cases as iconic, particularly retrospectively — will Twiqbal be such an icon in another few generations?

 

3

Why Do Peer Review?

poor-review-and-peer-review(Cross-posted at Prawfs, where I’m visiting this month.)

A recent post by Steve Bainbridge raises a nice issue: how should we think about peer review? Traditional peer-edited legal journals have established procedures (JELS pays honoraria and blinds; JLS pays but doesn’t; JLEO has fantastic peer comments, etc).But in the last five years, most of the top student-edited journals have moved to some kind of peer system – and many of us are now routinely asked, after a student-led process, to review for publication That peer review is never paid, and very often professors are asked to review for journals that have never accepted them. *cough. Yale Law Journal I love and hate you. cough*  That can frustrate even non-curmudgeons.  Why do it?

  1. For institutional credit. I’m aware of no school that gives formal credit for these student-edited peer reviews. Are you? If so, what does it look like?
  2. For Law Review credit. One explanation I’ve heard for doing a review for, say, Harvard Law Review, is to motivate them to feel  that they owe you at least a rejection on your own work, instead of a magnificent silence. In my experience, there’s some truth in this: doing peer review gives you the email of an AE, and credit with that person. I routinely have succeeded at being at least read by a journal I’d just done peer review with. I haven’t yet moved from a read to an acceptance.  But I did get a personalized email from HLR once.  It mentioned that they had an unusual number of great articles that cycle, which meant that they couldn’t publish even good work like mine.  I thought that was nifty! Of course, the credit isn’t  merely transactional: being a peer reviewer means you are an “expert” in the field, which should provide your article some kind of halo effect. Of course, this feeling is a quickly depreciating asset, and never rolls over from year-to-year. Use it or lose it!
  3. For the love of the game: For those of us who think that student journals should move exclusively to double-blind review, with faculty participation is a veto, participating is a price we should gladly pay. The problem is that the system isn’t perfectly constructed. Law journals should insist that peer comments will be conveyed to authors – this makes the comments much less likely to be petty (“cite me!”) and more likely to be constructive.

Bainbridge argues against mixed peer review systems, but none of his objections strike me as particularly relevant if the process is “student-screen, peer-veto.” That is how I understand the system to work at SLR, YLJ and HLR. I don’t know about Chicago – I would’ve thought their selection involves a maximizing formula and ended with a number.

389099659f
0

What Do Contract Formalities Do?

In 1941, Lon Fuller published his classic Consideration and Form. Among other things, the article articulated three famous functional claims for consideration: it evidences bargains, channels parties’ behavior, and cautions signers by “check[ing] against inconsiderate action,” and “induc[ing] the circumspective frame of mind appropriate in one pledging his future.”  How? By signaling to prospective signers of contracts that the law was drawing near.  Thus, he hypothesized that seals (“symbol[s] in the popular mind of legalism and weightiness”), the “requirement of a writing,” “attestation, notarization,” and recitals of consideration all induce individuals to feel and behave in a more committed way to the underlying term supported by the formal recitation.

I’ve been studying what individuals think about contract formalities in a series of papers.  That work, combined with other recent scholarship about contracting behavior, made me skeptical that contract language reciting obligation — or disclaiming it — had the straightforward effects that Fuller proposed.  So, with Zev Eigen (Northwestern/visiting Yale), I decided to test Fuller’s foundational & empirical intuition.  In A Fuller Understanding of Contractual Commitment, Zev and I suggest that the conventional account is unrealistic:

“Contract recitals are ubiquitous. Yet, we have a thin understanding of how individuals behave with respect to these doctrinally important relics. Most jurists follow Lon Fuller in concluding that when read, contract recitals accomplish their purpose: to caution against inconsiderate contractual obligation. Notwithstanding the foundational role that this assumption has played in doctrinal and theoretical debates, it has not been tested. This Article offers what we believe to be the first experimental evidence of the effects of formal recitals of contract obligation — and, importantly too, disclaimers of contractual obligation — on individual behavior. In a series of online experiments, we found that participants were less likely to back out of an agreement, forgoing personal gain, when they were endowed with a small extra sum of money at the time of contracting, and when they acknowledged that they were not forming a contract. They were more likely to back out of their original commitment when their agreeing was accompanied by a recital of consideration, and in a control condition in which the natural consideration of bargained-for exchange prevailed. Younger, male respondents were generally more likely to back out of their agreements across all conditions than were women and older participants. The reported experimental results suggest both the descriptive weakness of theorized accounts of private control over contract enforceability and the general value of experimental work about contracting behavior.”
The paper suggests that to the extent that we think formal devices permitting private party control over enforceability are useful, we might want to think carefully about developing ones that signal “law” more clearly to 21st century eyes.  I’d love to get your comments — the paper is still in draft form.

 

Yawning is a political strategy?
Yawning is a political strategy?
3

(Il)liberalism’s Paralysis

At Vox, Ezra Klein claims that the White House believes the bully pulpit drives illiberal dissensus on most matters of public concern.

“The problem is the White House no longer believes Obama can bridge [racial] divides. They believe — with good reason — that he widens them. They learned this early in his presidency, when Obama said that the police had “acted stupidly” when they arrested Harvard University professor Skip Gates on the porch of his own home. The backlash was fierce. To defuse it, Obama ended up inviting both Gates and his arresting officer for a “beer summit” at the White House . . .

Moreover, Obama’s presidency has seen a potent merging of the racial and political divides. It’s always been true that views on racial issues drive views on American politics. But as political scientist Michael Tesler has documented, during Obama’s presidency, views on American politics have begun driving views on racially charged issues.

This all speaks to a point that the White House never forgets: President Obama’s speeches polarize in a way candidate Obama’s didn’t. Obama’s supporters often want to see their president “leading,” but the White House knows that when Obama leads, his critics becomeeven less likely to follow. The evidence political scientists have gathered documenting this dynamic is overwhelming, and Frances Lee lays it out well here…”

Klein (channelling the White House) and others seems to suggest that this illiberal bully pulpit effect is a feature of the modern presidency.  I take it this argument comes in parts: (1) twitter & 24-hour news cycle require comments on every issue lest president be seen as passive and consequently weak; (2) modern media fragment the bully pulpit’s message and make it more likely for any speech to take on purely political valence; so therefore (3) presidential speeches will, through naive realism, harden battle lines and make it more difficult for non-political institutions to come to solutions.  Therefore, says the hyper-sophisticated Klein and other savvy consumers of our political science and psychology literatures, Presidential pablum is the future. While the White House may use rhetorical nudges on the margin, any attempt to move the needle on matters of public note is basically self-defeating.

This all strikes me as too clever by half. It takes the wrong lesson from problem of cognitive illiberalism (as popularized by the the Cultural Cognition Project’s blog).  It’s not that we are doomed to process information through cultural lenses, and that we inevitably view political leaders from opposite parties as our antagonists.  Rather, the way that messages are framedwhether threatening to identity or not – matters a great deal.  Or to put it differently, cultural dissensus isn’t inevitable. Look at nanotech and GM foods: notwithstanding all the preconditions for cultural warfare, the western front remains silent.

My intuition is that the President’s rhetorical boomerang effect doesn’t so much result from a structural feature of the modern Presidency as a bug. That bug flows from some set of small tics related to how the President speaks to audiences – how he (and his speechwriters) can’t manage to make it seem like his gestures toward opposing views are anything other than convenient.  (This is an empirical intuition and I’m more than open to the possibility that the “bug” also/instead turns on the President’s race.)

Worse, casting about for structural explanations is self-defeating & weak.  The Administration’s repeated public signaling that they know that the Presidency can’t move the needle creates a self-reinforcing feedback loop.  The more that the President says publicly that he lacks power to influence the world (because of SCIENCE!), the less power he actually has. It would be better to talk less about talking less.

3

Is Using USNews Ranking in Making Hiring Decisions Discriminatory?

That’s what Drexel 3L William Hanrahan claims in a newly filed complaint against Dechert LLP.  According to Hanrahan, though he ranks #4 in Drexel’s class, Dechert (and other large firms) hire disproportionately from higher ranked schools, and consequently refused to hire him.  Why is that a problem? Well, here’s what the complaint says:

screenshotHanrahan, who suffers from “Asperger’s Syndrome, an Autism Spectrum Disorder, and a concomitant non-verbal learning disability,” also argues that Drexel accepted more students than other local schools with disabilities, and that refusal to hire from Drexel tends to disproportionately screen out non-disabled candidates.

I’m not an expert in this area of the law, but I thought the complaint provided an interesting set of facts for discussion. My uninformed view is that the chain of causation (disability –> lower LSAT –> lower-ranked school –> fewer job offers) isn’t incredible, but that it’s hard to imagine a judge forcing firms to discount rankings (which, after all, aren’t entirely or even mostly based on student credentials) when making hiring decisions.

 

Any resemblance to current members of the SELS council is purely coincidental.
Any resemblance to current members of the SELS council is purely coincidental.
5

“Mighty In Their Day:” Reflections on the 9th Annual Empirical Legal Studies Conference

In Tolkein’s legendarium, the 9 rings of power were given to mortal men as a means of their corruption.

“Those who used the Nine Rings became mighty in their day, kings, sorcerers, and warriors of old. They obtained glory and great wealth, yet it turned to their undoing. They had, as it seemed, unending life, yet life became unendurable to them. They could walk, if they would, unseen by all eyes in this world beneath the sun, and they could see things in worlds invisible to mortal men; but too often they beheld only the phantoms and delusions of Sauron. And one by one, sooner or later, according to their native strength and to the good or evil of their wills in the beginning, they fell under the thraldom of the ring that they bore and of the domination of the One which was Sauron’s. And they became forever invisible save to him that wore the Ruling Ring, and they entered into the realm of shadows. The Nazgûl were they, the Ringwraiths, the Úlairi, the Enemy’s most terrible servants; darkness went with them, and they cried with the voices of death. — The Silmarillion, Of the Rings of Power and the Third Age, 346.

The fate of those holding one of the Nine struck me as a useful starting off point for my review of the Ninth Empirical Legal Studies Conference. [For previous installments in my CELS recap series, see CELS III,IV, V, and VI, VII, VIII.1, VIII.2]  The ring-of-power story is apt for several reasons.  ELS is waxing — we’ve obtained “glory and great [relative] wealth,” yet our methods are often described as inscrutable, as we see “things in worlds invisible to mortal men.” One well-known law blogger and sci-fi geek repeatedly has claimed that we behold only Sauron.  Ultimately, there’s a fairly decent argument, based on this year’s conference, that our thraldom — to machine learning — is nigh.  But putting aside the obvious parallels between the world’s leading legal empiricists and Angmar, the witch-king, there’s a far more pressing reason to use the 9 rings as a hook. Multiple sources told me that they found last year’s two-part recap to be “boring” or at best “workmanlike,” asking for more “made-up anecdotes” to spice it up.  So, off we go to Berkeley.

To start, let’s acknowledge the obvious. The West Coast is terribly distant from the home schools of most of the conference’s attendees. (I can’t prove that with data, but I thought this was exactly the kind of unsourced gossip that my readers wanted to see here.)  That was especially true for roughly 200 attendees from the Max Planck institute & the entire faculty of every Israeli law school.  The weather rendered the long trip tolerable, but only just.  Why not bend to reason and just hold a future conference in Germany or in Tel Aviv? Certainly, the conference is now decidedly more international in scope than it was only a few years back.  Was this the result of a maturing discipline, rapidly falling domestic travel budgets, or some unknown missing variable?  Other than the location, which they couldn’t help and probably were proud of, as West Coasters tend to be, the organizers (Anne Joseph O’Connell and Eric Talley) were magnificent and deserve credit for pulling off an enormous project without a hitch.

I arrived in time for both plenaries the first day. In a session on The Future of Big Data and Social Science, I learned that it’s much easier to do social science research when pesky IRBs don’t stand in your way. Though, given recent events, maybe IRBs only get in your way if you bother to tell them you are working on manipulating the political process with your purloined state seals.  In Evidence on Income and Wealth Inequality, Emannuel Saez pitched the utility of very high marginal tax rates as the (only?) solution to persistent and rising inequality. When pushed to articulate whether and how inequality was a social evil, he not surprisingly responded with a market-based argument: i.e., his co-author’s book sales demonstrated the issue’s political salience, and, consequently, the question’s irrelevance.  I thought this was a rather chippy answer, though at the end you have to give it to him.  It may be the least read popular book of the last fifty years, but that’s a ways better than the least read unpopular book!

Read More

0

Introducing Guest Blogger Jeffrey Vagle

jvagleWe are happy to host Penn’s Jeff Vagle for the month of November.

Jeff is Lecturer in Law and Executive Director of the Center for Technology, Innovation and Competition at the University of Pennsylvania Law School. His research interests include cryptography, cybersecurity, electronic privacy, the mechanisms and societal effects of surveillance, Internet architecture, and networked economies and societies. He most recently served as an associate in Pepper Hamilton’s Privacy, Security and Data Protection Group. He earned his JD from Temple University School of Law, where he was Editor-in-Chief of the Temple International and Comparative Law Journal. Jeff writes and speaks regularly on privacy, data security, surveillance, and other cyberlaw-related topics, and is the author of several law review and technical articles, including, most recently, “Furtive Encryption: Power, Trust, and the Constitutional Cost of Collective Surveillance,” forthcoming in the Indiana Law Journal.