Category: Symposium (What Difference Representation)


Mandatory (Randomized) Clinical Trials For Law Schools

Clinics Should Take Their Chances

I’m really proud that Concurring Opinions is hosting a symposium today and tomorrow on What Difference Representation.  I think the piece offers a timely and provocative thesis, and we’ve assembled a near-ideal group of folks to engage with the topic.  I look forward to seeing what the group comes up with.  A big thank you to Jaya for organizing with me, and of course to Jim and Cassandra for agreeing to hold up their work to public scrutiny.

I’ve been thinking over the last several weeks about what I could say that would be distinctive. Here’s what I came up with.  That selection makes hay of our ability to study lawyering in the wild is obvious – and Representation’s focus on the hazards of previous work in the field an important contribution.  Generally, selection–  in various guises — produce one of empirical legal studies’ true wicked problems. It shocks me how many quantitative and qualitative papers proceed as if it it were a molehill to dismiss, instead of a mountain to climb. Randomized trials promise a solution — really, the only way we can know if legal policies are having the effects that we hypothesize.

Law school clinics offer one of the few fora where randomized trials would be ethically & practically feasible.  It is true that no law school clinic experiment can fill the offer/representation gap that bedevils the paper.  But comparison across clinics might shed light on the effect of resources, client characteristics, clinic structure, and legal regime on the offer-making outcomes that the paper discusses.  (Whether you a comparison of like-clinics would enable a better estimate of the representation effect than the methods employed on pps. 39-42 is an open question.)

Given that more data would permit better resourcing decisions, should law school clinics be expected to engage in randomized trials when offering representation? My tentative answer to this question is yes. Law school clinics are subsidized by student tuition dollars (and sometimes the public too).  This doesn’t and shouldn’t mean that pedagogical clinics should refrain from zealous advocacy.  But it does suggest, I think, that their mission should be informed and guided by the values of the University: openness to criticism, the expectation that participants will produce generalizable and socially useful knowledge, and measured transparency. Those values would be served if clinics participated in randomized testing to see if, and how, they are helping the recipients of their services.  Like Abramowicz, Ayres and Listokin, I endorse randomization before determining the effect of legal policy.  To my knowledge, not one law school clinic in the country engaged in  a randomized trial before engaging in lawyering services.  Doctors no longer practice this way – lawyers shouldn’t either.

It would be much better if this participation in research occur voluntarily, without resort to our accrediting agency, or, worse, the politicized and meddling hands of various state-related funding bodies.  I think moral suasion is the best route – hence, this blog post.  We’re anticipating that many clinicians will read this symposium.  To them I pose the following challenge: why not randomize and see what effects your offers are having on client outcomes?  Maybe, in the aggregate, we’ll learn something about representation.

While you are mulling that over that modest proposal, I hope you will enjoy the rest of the symposium.


What Difference Representation: Introduction to the Symposium

Should Law School Clinics Select Clients by Roulette?

I am delighted to announce that Concurring Opinions will be hosting a symposium next Monday and Tuesday on What Difference Representation? Offers, Actual Use, and the Need for Randomization, the forthcoming Yale Law Journal article by Jim Greiner and Cassandra Wolos Pattanayak.  [Update: You can read all posts in the symposium by clicking on this link.]  As you may recall, What Difference has already caused quite a stir in the clinical and legal aid communities. Given our shared interest in questions of empirical methodology, and Jaya’s background in clinical legal services, we decided that bringing that debate to CoOp would be an excellent use of our time and energy. Here’s the (revised) abstract) – though you should download the article if you haven’t already:

“We report the results of the first in a series of randomized control trials designed to measure the effect of an offer of, and the actual use of, legal representation. The results are unexpected. In the context of administrative litigation to determine eligibility for unemployment benefits, a service provider’s offer of representation to a claimant had no statistically significant effect on the claimant’s probability of a victory, but the offer caused a delay in the proceeding. Because a substantial percentage of the provider’s client base consisted of claimants who were initially denied benefits but who would later have that initial denial reversed as a result of the litigation, the offer of representation inflicted a harm upon such claimants in the form of an additional waiting time for benefits to begin, this with no discernible increase in the probability of a favorable outcome. In other words, within the limits of statistical uncertainty, these claimants would have been better off without the offer of representation. The size of the delay (around two weeks, depending on how measured) was not large in absolute terms, and would have been negligible in many other legal settings, but was relevant in the context of this particular administrative and legal framework, one in which speed has remained a special concern for decades. Moreover, in a small number of cases with a certain profile, the delay caused the unemployment system to continue paying benefits erroneously for a longer period of time, potentially imposing costs on the financing of the unemployment system. We were also able to verify a delay effect due to the actual use of (as opposed to an offer of) representation; we could come to no firm conclusion on the effect of actual use of representation on win/loss.

We hypothesize three potential explanations for our findings (and acknowledge that others are possible). First, it is possible that the client base that reached out to the service provider (and thus was subject to randomization) was a specialized subset of unemployment claimants, a subset that did not actually need legal assistance. This theory suggests special attention to provider intake systems. Second, it is possible that the administrative adjudicatory system at issue, with its semi-inquisitorial style of judging, is pro se friendly. Third, it is possible that the subject matter in dispute in these cases is less legally, factually, or procedurally complex than in other settings.

We caution against both over- and under-generalization of these study results. We use these results as a springboard for a comprehensive review of the quantitative literature on the effect of representation in civil proceedings. We find that this literature provides virtually no credible information, excepting the results of two randomized evaluations occurring in different legal contexts and separated by over three decades. We conclude by advocating for, and describing challenges associated with, a large program of randomized evaluation of the provision of representation, particularly by legal services providers.”

We have assembled a terrific group of symposiasts, mixing clinicians, academic empiricists and practitioners. Besides Jim, Cassandra, Jaya and me,  the group includes twelve contributors, lauded in detail after the jump:

Read More