What Would Happen if USNews Didn’t Weigh Money?

no-money-300x300-150x150Recently the ABA announced that it will no longer collect expenditures data from law schools: Leiter and Merritt offer thoughts on how that decision will influence the USWR rankings.  Both posts are interesting, though somewhat impressionistic.  Leiter thinks that state schools will benefit and Yale will lose it’s #1 spot; Merritt believes that USWR should reconfigure its method. [Update: Bodie adds his two cents.]

It’s well known that the influence of particular categories of data on the ranking can’t be determined simply by reading the charts that the magazine provides. Paul Caron notes that the rankings depend on on inputs that aren’t displayed (like expenditures). But it gets worse: (1) the point accumulation of each school influences that of every other school; (2) USWR changes the raw data through manipulations that are not well explained (placement discounts for law school funded positions) or are simply obscure (CoL adjustments for expenditures); (3) many schools don’t report information and USWR doesn’t advertise their missing-data imputation method; etc. etc. Bottom line: the rankings are very, very fragile.  (Many would say they are meaningless except at 10,000 feet.)  Luckily, Ted Seto’s work enables everyone to give their best shot to approximating each year’s ranking.  Seto argues that variance within a category turns out to influence the final scores as much as the purported weight that USWR assigns to it.

As thought experiment, I decided to estimate what would happen if each school’s expenditure data was set to average school’s expenditure.  I then used Seto’s method on 2011-2012 historic data to estimate the rankings in the absence of expenditure variance.  This basically eliminates the influence of expenditure as a category.  (A perhaps better, but more time consuming, approach would be to eliminate the expenditure categories altogether and re-jigger the equation accordingly). My back-of-the-napkin approach produces some wacky results, particular at the lower end of the ranking scale.  To keep it simple, after the jump I’ll focus on the top ten winners and losers from the elimination of expenditure variance in the 2013 t100 and then offer some thoughts.

Current T100 Schools with the largest estimated gains from the absence of expenditure variance:

1.  University of Kentucky

2.  Brigham Young University

3.  Washington and Lee

4.  University of Alabama

5.  University of Louisville

6.  Cornell

7. University of Arizona

8.  University of Utah

9. College of William and Mary

10. University of Arkansas

Current T100 Schools with the largest estimated losses from the absence of expenditure variance:

 1. Brooklyn

2. Syracuse

3. American

4.  Seton Hall

5. Santa Clara

6.  Miami

7. Penn State

8.  Yale

9.  DePaul

10.  Marquette

And...Leiter is basically right.  Some state schools with relatively good employment outcomes and relatively low expenditures per student would do better in the absence of any expenditure data; private & stand-alone schools (who were benefiting from the absence of indirect expenditures) will lose out. Notably, none of the effect sizes I saw was huge – the largest would produce a 5 – 10 point swing in the ultimate rank of the school.

You may also like...

10 Responses

  1. Jimbino says:

    You say, “I decided to estimate what would happen if each school’s expenditure data was set to average school’s expenditure.”

    I think you and other legal writers should master use of the subjunctive mood. The “would” indicates that use of a verb in the subjunctive will be necessary in the subordinate clause beginning with “if”.

    Your use of indicative “was set” instead of subjunctives “had been set” or “were set” is what leads to confusion. As it is, your statement could mean either:

    1. I decided to estimate what would happen if each school’s expenditure data had in the past been set to average school’s expenditure.


    2. I decided to estimate what would happen if each school’s expenditure data were now set to average school’s expenditure.

    You will argue that the sense is clear from the context, but you will then have to take responsibility for your readers’ having to re-read the paragraph to try to get your gist.

    Don’t worry, even self-satisfied “legal grammarian” Eugene Volokh makes all kinds of grammatical errors, and regularly. He says, “at risk for” when he means “at risk of,” and he uses “forbid from” when he means “forbid to,” evidently confusing idiomatic usage of “forbid” with that of “prohibit.”

  2. Dave Hoffman says:

    I would love it if I’d be able to master the subjective mood.

  3. Rick Underwood says:

    I assume that you have heard the joke about the seafood loving lawyer who asked the cab driver in Boston where he could get scrod.

  4. Rick Underwood says:

    So the cabdriver was a Classics Major who could not find a job involving his degree – so he was driving a hack. He was thinking about going to Law School at the University of Kentucky. He responded to the lawyer, “That’s the first time that I have been asked for it in the past pluperfect subjunctive.

  5. Mike says:

    Understanding that your estimates are back-of-the-napkin, and that you’ve chosen only a crude measure of the effect of eliminating the category, and that you may plant another stylistic error to inflame the masses and generate traffic . . . it would be interesting to see more of what your calculations indicated, either in terms of the magnitude of movement or additional schools.

  6. Kipper says:

    Where can one find the 2011 data which you used?

  7. Alan A. says:

    Dave Hoffman writes,

    “I would love it if I’d be able to master the subjective mood.”

    At your present level of both grammar and wit, “mastery” would be a tall order. How about simply learning what the subjunctive mood is, first? See below, and feel free to thank me, ungrammatically, in the comments:

    http://www.dummies.com/how-to/content/using-the-subjunctive-mood-in-english.html .

  8. Dave Hoffman says:

    Alan A.

    Thank you, ungrammatically.

  9. dave hoffman says:

    Kipper –

    Read Seto’s article. It’s a very good roadmap.

  10. Jull says:

    I am sure ranking would not change much, especially when taking into consideration the criteria. The quality and the price of education is what is going to matter, for making choice in the sense of education establishment to apply, especially with the rising student loan rates.
    Have you mastered oblique moods, by the way, Dave?