Author: Jim Greiner


How Much Enthusiasm for Randomized Trials? A Response to Kevin Quinn and David Hoffman

We thank Kevin Quinn and David Hoffman for taking the time to comment in our paper.  Again, these are two authors whose work we have read and admired in the past.

Both Dave and Kevin offer  thoughts about the levelof enthusiasm legal empiricists, legal services providers, and clinicians should have for randomized studies.  We find ourselves in much but not total agreement with both.  To Kevin, we suggest that there is more at stake than just finding out whether legal assistance helps potential clients.  In an era of scarce legal resources, providers and funders have to make allocation decisions across legal practice areas (i.e., should we fund representation for SSI/SSDI appeals or for unemployment appeals or for summary eviction defense).  That requires more precise knowledge about how large representation (offer or actual use) effects are, how much bang for the buck.  Perhaps even more importantly, scarcity requires that we learn how to triage well; see Richard Zorza’s posts here and the numerous entries in his own blog on this subject.  That means studying the effects of limited interventions.  Randomized trials provide critical information on these questions, even if one agrees (as we do) that in some settings, asking whether representation (offer or actual use) helps clients is like asking whether parachutes are useful.

Thus, perhaps the parachute analogy is inapt, or better, it requires clarification:  we are in a world in which not all who could benefit from full-service parachutes can receive them.  Some will have to be provided with rickety parachutes, and some with little more than large blankets.  We all should try to change this situation as much as possible (thus the fervent hope we expressed in the paper that funding for legal services be increased).  But the oversubscription problem is simply enormous.  When there isn’t enough to go around, we need to know what we need to know to allocate well.  Meanwhile, randomized studies can also provide critical information on the pro se accessibility of an adjudicatory system, which can lay the groundwork for reform.

To Dave, we say that our enthusiasm for randomized studies is high, but perhaps not high enough to consider a duty to randomize among law school clinics or among legal services providers.  We provided an example in the paper of practice in which randomization was inappropriate because collecting outcomes might have exposed study subjects to deportation proceedings.  We also highlighted in the paper that in the case of a practice (including possibly a law school clinic) that focuses principally on systemic change, randomization of that practice is not constructive.  Instead, what should be done is a series of randomized studies of an alternative service provider’s practice in that same adjudicatory system; these alternative provider studies can help to assess whether the first provider’s efforts at systemic change have been successful.

Our great thanks to both Kevin and Dave for writing, and (obviously) to Dave (and Jaya) for organizing this symposium.


Randomized Trials Do Not Require Abandoning Merits Screens Or Ethical Duties: A Response to Margaret Monsell

To a hypothetical legal services provider that objects to a proposed randomize study on the grounds that participating would compel it to represent “clients with unmeritorious claims” (see Margaret Monsell post below), we ask the following question:  if someone won a lottery and offered to give that provider an extra $100,000 to represent additional cases in a practice area, would the provider turn the money down on the grounds that doing so would compel it to represent clients with unmeritorious cases?  In other words, is the match of resources to caseload perfect, or nearly perfect, as matters currently stand, or are there practice areas in which a great many clients who would benefit from legal assistance in fact go unrepresented?

We very much appreciate Margaret Monsell’s taking the time to comment on our paper.  One of the most encouraging and informative aspects of our current research is that we have had the opportunity to interact with persons in both the front line and, now, the “backup” aspects of legal services delivery and adjudicatory system improvement.

We write to clarify that that randomized trials are entirely consistent with merits screens and with lawyers’ duties of candor to the court.

Read More


Fair Criticism: A Response to Rebecca Sandefur, Andrew Martin, Michael Heise, and Ted Eisenberg

We very much appreciate the time Rebecca Sandefur, Andrew Martin, Michael Heise, and Ted Eisenberg have taken to comment on our paper.  We are particularly excited by comments from these authors because for each, we have read and admired his/her work in the past.

We believe that much of the criticism expressed in these comments is well-taken, and we will react accordingly.

Read More


The Centrality of Abstracts? A Response to Bob Sable’s and David Udell’s Comments on “What Difference Representation? Offers, Actual Use, and the Need for Randomization”

Our great thanks to David Udell and to Bob Sable for taking the time to comment (separately) on our paper, “What Difference Representation?  Offers, Actual Use, and the Need for Randomization.”  We very much appreciate the comments they have made, and as we hope is clear from the introduction and elsewhere in our paper, we have the greatest respect for the work that they do, the work that their organizations do, and the legal services community to which they belong does.

Some uncomfortable aspects of writing this paper are that we find ourselves sometimes disagreeing with persons and organizations we greatly admire, being held responsible for what an advocate did not include in a legal brief, and having our study implicitly compared to the Gingrich Congress’ efforts to limit legal services funding and to a false expose seeking to tar legal services programs.  Had David’s and Bob’s criticisms concerned what we said in our paper, we might have considerable cause for regret.  As we understand them, however, the lion’s share of David’s and Bob’s comments go not to the content of the paper but to the title and abstract.  There are some substantive points, to which we respond below, but the primary thrust of both comments is that we have been reckless in the title and the abstract by not including and highlighting caveats that David and Bob for the most part agree that we discuss in the paper’s text.

We wonder about the apparent centrality of titles and abstracts.  But conceding that point for the moment, we also wonder whether the sins of omission and selective emphasis David and Bob accuse us of committing apply to their blog posts.  By way of example, none of the following appears in either of their posts:  (i) that the full title of the paper (which appears above) references the distinction between offers and actual use; (ii) that the first sentence of the abstract says that our research program is “designed to measure the effect of an offer of, and the actual use of, legal representation”; (iii) the last sentence in this paragraph in the abstract, after again referencing “the actual use of (as opposed to an offer of) representation,” reports that “we could come to no firm conclusion on the effect of actual use of representation on win/loss”; (iv) the third sentence of the paper again references “both an offer of, and actual use of, representation”; and (v) Part B of the introduction dedicates several pages discussing the distinction.  We have expanded the abstract several times already in response to concerns from legal services providers (including HLAB itself), and we will consider doing so again, but perhaps the best thing to do at this point would be simply to omit the abstract entirely.  We will consider that as well.

To substance.

Read More