Judged by the Company You Keep
Last week, I tried to outline the difficulties associated with measuring judicial ideology in regards to the limited alternatives that have been offered by scholars. In this post, I hope to describe how I have measured it and attempted to overcome the various obstacles brought about by my methodology.
My idea for identifying the ideologies of federal appellate judges was to determine the rates at which such judges agree and disagree with “conservatives” and “liberals” on the bench. The assumption was that like-minded judges will vote together more often and judges with dissimilar ideologies will tend to disagree. By focusing on the agreements and disagreements among the judges, the goal was to pinpoint their respective ideologies (via “ideal points”). This is an agnostic method that necessarily faces all of the shortcomings of such an approach that I previously described.
The initial concern with such a method is that there are far too few disagreements among the judges on the Courts of Appeals. Indeed, in the 10,242 cases in my dataset, there were only 288 dissents (including partial dissents). Some judges who participated in over 100 cases were not on a panel in which there was a single dissenting vote. Looking at the Courts of Appeals alone was, thus, unlikely to offer much information. My solution was to treat the district judges being reviewed as pseudo-fourth members of the appellate panel. After all, the district judge reviewed the same legal issue as the appellate panel and rendered judgment on that very same issue. Notably, there are far more disagreements with district judges in the form of reversals. Also, by including the district judges, my methods also allowed data to be harvested from unanimous affirmances as well (as described below).
However, my solution faced numerous difficulties which necessitated adding nuance to the general concept for the measure. Importantly, I incorporated these details into my Ideology Scores:
- Case Mixes – different judges hear different types of cases. In my dataset, the most significant difference that was relevant to the measure was in relation to criminal and civil cases. District judges were reversed twice as often in civil cases. Accordingly, each judge (or other unit of measure) was analyzed assuming an average distribution of civil and criminal cases.
- Standards of Review – because appellate judges review district court judges with deferential and non-deferential standards of review, there is a need to adjust for standards used. For example, a reversal in a criminal case with a deferential standard of review is far less common than a reversal in a civil case with a non-deferential standard. So, in addition to the case mix adjustment, judicial votes were further weighted based upon the standard of review used.
- Rates, not Frequencies – judges have differing co-panelists and judges reviewing their decisions. As a result, it was essential to focus on the percentages of agreements rather than the absolute numbers. So, judges were evaluated for the percentage with which they agreed with judges appointed by Republicans and by Democrats in each of the relevant sub-categories (i.e., criminal case with a non-deferential standard). The use of rates also provides a stable baseline for comparisons so that judges who are in the minority ideology on their court are measured in the same manner as those in the majority group. Rates also incorporate affirmances into the measure – something that previous agnostic measures have been unable to do.
- Panel Effects – one of the most significant results in recent empirical legal studies has been that the composition of the panel determines to a large degree how an individual judge votes. Judges who sit with more Republican appointees will tend to vote in a more conservative direction than if they sat with Democratic appointees. Accordingly, the measure was adjusted to assume that judges had the same co-panelists. This was done by determining the differential in co-panelists arrangements between two Republican appointees and two Democratic appointees.
- Inter-Circuit Adjustments – the Courts of Appeals in my dataset have their jurisdiction determined by geography. As a result, they review differing sets of cases and differing district judges. This would tend to make inter-circuit comparisons impossible. However, the circuits are not entirely insular. Judges who have taken senior status often travel to different circuits and sit by designation. In my dataset, there were 2,472 votes issued by 26 “traveling judges” in either their home or away circuits. Using data from how those judges voted at home and away, adjustments were made to the Ideology Scores of judges on each circuit. This adjustment assumed that a judge remains constant in his or her ideology regardless of circuit. Again, accounting for all of the above adjustments, it was determined whether a particular circuit made travelling judges vote more liberally or conservatively.
Based upon all of those adjustments, individual Ideology Scores were determined for every federal appellate judge and district judge who issued an opinion in 2008 or had their opinion reviewed in 2008. However, due to sample size concerns, the article focused on 138 judges who sat on the Courts of Appeals that year. In my next post, I will discuss how my measure does against the two leading measures. Below are some example scores (in comparisons to the other leading measures) from notable judges studied in my article. Total interactions refers to the total number of co-panelists and district judges included the sample. Higher scores are more conservative and lower scores are more liberal.
|Judge||Circuit||Total Interactions||Party of Appointing President||Common Space Score||Ideology Score|
|Cook, Deborah L.||6||178||Rep.||0.226||77.2|
|O’Scannlain, Diarmuid F.||9||270||Rep.||0.023||59.7|
|Easterbrook, Frank H.||7||241||Rep.||0.559||55.8|
|Jones, Edith H.||5||291||Rep.||0.502||22.0|
|Boggs, Danny J.||6||156||Rep.||0.339||17.3|
|Thomas, Sidney R.||9||341||Dem.||-0.209||-5.4|
|Posner, Richard A.||7||254||Rep.||0.034||-9.9|
|Reinhardt, Stephen R.||9||143||Dem.||-0.409||-23.3|
|Williams, Ann C.||7||253||Dem.||-0.345||-31.5|
|Wood, Diane P.||7||277||Dem.||-0.3795||-37.2|
|Wardlaw, Kim M.||9||255||Dem.||-0.338||-63.3|