Brian Tamanaha’s Straw Men (Overview)
(Cross posted from Brian Leiter’s Law School Reports)
Brian Tamanaha previously told Inside Higher Education that our research only looked at average earnings premiums and did not consider the low end of the distribution. Dylan Matthews at the Washington Post reported that Professor Tamanaha’s description of our research was “false”.
In his latest post, Professor Tamanaha combines interesting critiques with some not very interesting errors and claims that are not supported by data. Responding to his blog post is a little tricky as his ongoing edits rendered it something of a moving target. While we’re happy with improvements, a PDF of the version to which we are responding is available here just so we all know what page we’re on.
Some of Tamanaha’s new errors are surprising, because they come after an email exchange with him in which we addressed them. For example, Tamanaha’s description of our approach to ability sorting constitutes a gross misreading of our research. Tamanaha also references the wrong chart for earnings premium trends and misinterprets confidence intervals. And his description of our present value calculations is way off the mark.
Here are some quick bullet point responses, with details below in subsequent posts:
- Forecasting and Backfilling
- Using more historical data from SIPP would likely have introduced continuity and other methodological problems
- Using more years of data is as likely to increase the historical earnings premium as to reduce it
- If pre-1996 historical data finds lower earnings premiums, that may suggest a long term upward trend and could mean that our estimates of flat future earnings premiums are too conservative and the premium estimates should be higher
- The earnings premium in the future is just as likely to be higher as it is to be lower than it was in 1996-2011
- In the future, the earnings premium would have to be lower by **85 percent** for an investment in law school to destroy economic value at the median
- Data sufficiency
- 16 years of data is more than is used in similar studies to establish a baseline. This includes studies Tamanaha cited and praised in his book.
- Our data includes both peaks and troughs in the cycle. Across the cycle, law graduates earn substantially more than bachelor’s.
- Tamanaha’s errors and misreading
- We control for ability sorting and selection using extensive controls for socio-economic, academic, and demographic characteristics
- This substantially reduces our earnings premium estimates
- Any lingering ability sorting and selection is likely offset by response bias in SIPP, topcoding, and other problems that cut in the opposite direction
- Tamanaha references the wrong chart for earnings premium trends and misinterprets confidence intervals
- Tamanaha is confused about present value, opportunity cost, and discounting
- Our in-school earnings are based on data, but, in any event, “correcting” to zero would not meaningfully change our conclusions
- Tamanaha’s best line
- “Let me also confirm that [Simkovic & McIntyre’s] study is far more sophisticated than my admittedly crude efforts.”