US News Rankings – A Different Role for Inputs
Instead of our current world where higher LSAT/GPA numbers lead to more choices, I want to consider a hypothetical admissions model where students are randomly assigned to schools and forced to attend the schools selected for them. In such a world, LSAT/GPA scores would be relatively equal across all schools (in comparison to the present distribution). In formulating a rankings system from a student perspective, we would only care about what the schools do to improve the relatively equal incoming student quality (measured by job placement or other output variables).
Still, particularly for smaller schools, there would be some statistical variation in incoming LSAT/GPA scores and we would seek to control for those differences in assessing the output variables such as job placement. An ideal rankings system would discount success that could be attributed to incoming student quality and vice versa for schools with lower incoming scores. So, in a world where variations between incoming classes are small and unlikely to have substantial effects on the overall rankings, a good statistician would still like to control for the expected variation in class quality. But, strangely, in a world where the variations in entering class quality are very large, US News not only doesn’t control for entering class quality, it actually adds it to its overall formula in a prominent manner. This makes little sense if the goal is truly to measure law school quality (however it is defined) and aid students in their decision-making.