Category: Education

0

Measuring Law School Teaching (Continued)

Evstafiev-barocoa-school.jpgThanks to various commentators (and Professor Carrell) for sending me this paper measuring the effect of teaching on students’ performance. As I’ve previously written, the paper (and its related literature) provides reasons to question assumptions about the relationship between student satisfaction and learning.

Two paragraphs from the paper, which I’ve now read, stand out:

Results show the less experienced and less qualified(by education level) calculus professors produce students who perform better in the contemporaneous course being taught, however, these students perform significantly worse in the follow-on advanced mathematics-related courses. Although, we can only speculate as to the mechanism by which these effects operate, one might surmise that the less educated and experienced instructors may teach more strictly to the regimented curriculum being tested, while the more experienced professors broaden the curriculum and produce students with a deeper understanding of the material. This deeper understanding results in better achievement in

the follow-on courses.

Ah, we’ve finally found a good potential defense of tenure: it enables brave souls to deviate from bad, top-down, curricular mandates! This section highlights a key difference between law school teaching and the undergraduate instruction studied in the paper: law school teaching isn’t generally guided by a centrally-administered curriculum. Unlike instructors at the college level, law professors are almost never told at any level of useful detail what their courses should cover. (I’ve heard odd rumblings that even attempts to coordinate instruction across sections would impinge on academic freedom!) This difference makes it hard to study law school teaching across professors, and makes it all but impossible to replicate this study in the law school classroom.

Assuming, however, that the study has general implications, the finding about teaching evaluations is of particular interest:

[P]rofessor evaluations in the initial courses are very poor predictors of student achievement in the follow-on related courses. Of the 27 coefficients [of achievement studied] 13 coefficients are negative and 14 are positive, with none statistically significant at the 0:05 level. Again, results for question 22, which asks students, “Amount you learned in this course was:” show that a 1-point (equivalent to 1:8 standard deviations) increase in the mean professor evaluation resulted in a statistically insignificant .014, -.008 and -.018 respective standard deviation change in calculus, science, and humanities follow-on related course achievement. Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this finding draws into question the value and accuracy of this practice.

A commentator to my previous post suggested that this result suggests a problem in the grading system. I agree! Professors aren’t being graded by students on indices relevant to how well the students are learning. Jason Solomon’s use of self-reported student gains in analytic ability scores in measuring “educational quality” thus appears to potentially mislead about how well law schools are doing.

I’ve suggested in the past that we ought to look for alternative ways of measuring teaching that go beyond student satisfaction. A nice approach, if the data were available, Or, as Bill Henderson (and others) have argued, you could focus on employment outcomes. Either of these alternatives, it seems to me, would dominate over student satisfaction metrics, which are (at best) a very bad proxy for whether law professors are doing their job, which is to model & instill in students a lawyer’s situation sense.

(Photo Credit: Wikicommons)

8

Boys, Girls, and Math

An study appearing in Science (no free access) apparently finds “There’s no real difference between the scores of U.S. boys and girls on common math tests.” (this link is to Science’s reporting about the article). Some interesting points include “Overall, the researchers found “no gender difference” in scores among children in grades two through 11.” and data that “suggests that cultural and social factors, not gender alone, influence how well students perform on tests.” On the SAT, however, boys out performed girls. The study argues that is because more girls take the test which obscures the results.

The part of the report that may be of most use to law and policy folks concerns No Child Left Behind:

Using a four-level rating scale, with level one being easiest, the authors said that they found no challenging level-three or -four questions on most state tests. The authors worry that means that teachers may start dropping harder math from their curriculums, because “more teachers are gearing their instruction to the test.”

As a side note, the first thing I thought of was the Barbie “I hate math” moment of more than fifteen years ago. A quick search of Barbie and math brought up possibly 466 news article that linked that event to the study (and the few I checked were not all AP wire coverage). In addition, Lawrence Summers’ statements about intrinsic aptitude pop up quite a bit in coverage of the study. So would the study have made such a splash without that history? As for Barbie, maybe someone will point out that its image is not as stable as Mattel might wish (whether that should matter is a different question depending on the legal context). And Mr. Summers, well this link seems to be his speech about the aptitude issue. Draw your own conclusions.

Inequality as a Political Phenomenon

When the great and good talk about inequality, they often presume that at its root is a skills differential between rich and poor. The former are supposedly much more educated than the latter, and this explains income gaps.

There are certain situations where returns to education are clear. However, a growing body of work makes it clear that political decisions are a key part of the equation as well. Consider Larry Bartels’s recent work (here reviewed by Dan Balz):

[T]here is a partisan pattern to the size of the gap between the rich and the poor. Over the past half-century, he concludes, Republican presidents have allowed income inequality to expand, while Democratic presidents generally have not.

Lest anyone think this book is a partisan hit job by a left-wing academic, Bartels goes to great pains in his introduction to preempt the counterattack he expects from critics on the right. “I began the project as an unusually apolitical political scientist,” he writes, noting that the last time he voted was in 1984, “and that was for Ronald Reagan.” He adds that in doing this work, “I was quite surprised to discover how often and how profoundly partisan differences in ideologies and values have shaped key policy decisions and economic outcomes. I have done my best to follow my evidence where it led me.”

No wonder “more than three-fourths (77.2 per cent) of US workers say they feel unrepresented by the political system on workplace issues.” They are likely becoming ever more skeptical of the “education remedy” for addressing inequality. Until the Neal Stephensonian nano-revolution is complete, we can’t all make our daily bread as manipulators of symbols. Adequate wages and health care for participants in the real economy–whatever their level of education–are a must.

1

The Citizenship Exam and Popular Culture

Every year as part of my constitutional law final exam, I give my students a mini “citizenship exam.” I ask six questions that derive from the U.S. Citizenship Exam, which are about the U.S. Constitution. I tell my students they can go to the U.S. government website and review the questions there, or they can just read the Constitution. Doing either should prepare them well for that portion of the exam. The questions I ask are straight forward and, I would suppose, easy: How does one amend the Constitution per Article V? What are the first ten amendments called? Who, according to the Constitution, has the power to declare war? Most students score 100% on this portion of the final exam — as I would hope. But sometimes I am surprised, unhappily so, by a pattern I see in the wrong answers. I asked the last question — the power to declare war — this year, and I was worried and angry by the number of students who answered that question: the President. (Angry: Weren’t they paying attention? Worried: What kind of teacher am I?) And then I thought about the question and how, as with some of the questions in the past, the right answer conflicts with our present experience of our political order and popular culture.

Of course the Constitution does say that Congress declares war, not the President, but since World War II, the United States Congress hasn’t declared war on any nation despite having authorized troops to serve in many, many war zones. No wonder students are confused. All the more reason they should have heard and remembered the classes on the war powers, I think. And all the more reason I should have been that much clearer in class. So although popular cuture can wreak havoc on learning, it can, of course, also be a teacher’s best friend. I tend to embrace it in my classroom (playing Billy Bragg’s Everywhere when I teach Korematsu, showing clips from West Wing when teaching Roe v. Wade). But in this case, I forgot the lesson. For our unit on war powers, I should have played this clip of President Bush declaring “mission accomplished” on May 1, 2003. Nothing like the problem staring you in the face to jump start a classroom discussion on the separation of powers. (And the problem can be defined in any number of ways for a good discussion about the constitutional order — the scope of the implied Art II powers, Congress’s reluctance to declare war but its willingness to fund it, etc.). Playing this clip by MoveOn.org would have just derailed the pedagogical lesson (so I will try and refrain next year) but perhaps it would good in some class. Suggestions welcome.

Quality and Equality in Education

A regular reader of the blog alerted me to this story on the Finnish education system, which has been studied around the world for its extraordinary accomplishments:

[B]y one international measure, Finnish teenagers are among the smartest in the world. They earned some of the top scores by 15-year-old students who were tested in 57 countries. American teens finished among the world’s C students even as U.S. educators piled on more homework, standards and rules.

Certainly cultural factors play an important role here. But Finland’s experience should also challenge any easy assumptions that pursuing equality in educational opportunity leads to a decline in the quality of education for top students:

Each school year, the U.S. spends an average of $8,700 per student, while the Finns spend $7,500. Finland’s high-tax government provides roughly equal per-pupil funding, unlike the disparities between Beverly Hills public schools, for example, and schools in poorer districts. The gap between Finland’s best- and worst-performing schools was the smallest of any country in the PISA testing. The U.S. ranks about average.

Finnish students have little angstata — or teen angst — about getting into the best university, and no worries about paying for it. College is free. There is competition for college based on academic specialties — medical school, for instance. But even the best universities don’t have the elite status of a Harvard.

Note that the same “lessening of the stakes” pervades much of the rest of the economy in Finland. America may need to reconsider whether its own inequalities in educational opportunity are not simply a burden on the disadvantaged, but also militate against our economic position as a whole.

1

And Now a Word From Our Sponsors…: The Ethics of Sponsored Courses and Maybe Chairs?

dollars2.jpgInside Higher Ed details that Hunter College offered a course that was sponsored by an industry group called International Anticounterfeiting Coalition (known as the IACC). The group represents major fashion industry companies. The class well that is where the fun begins. Apparently the

students would create a campaign against counterfeiting in which they would create a fake Web site to tell the story of a fictional student experiencing trauma because of fake consumer goods. One goal of the effort was to mislead students not in the course into thinking that they were reading about someone real.

The article raises some good questions: Why have students perform free labor for the fashion industry (and really pay for the privilege?)? What about the underlying lies? These issues remind me of the LonleyGirl issues (there a fake videoblog lured people into what appeared to be a true personal site but was a front for a group launching a film company. Eric Goldman has a set of quick links that highlight the problems of user-generated content, ads, and quality. In general the school’s willingness to offer a class that propagates a shall we say less than authentic Web site is an example of the marketer’s will. Not that this point should exonerate the school. (Note that apparently Iowa turned down money when it was unsure about naming a school after the donor).

Still according to the article “other colleges do work with IACC” including Ohio State University but at least Ohio State does not operate in the same way as Hunter allegedly did. Ohio State seems to set up the projects as out of class activities. Hunter’s class according to some was directed by the IACC such “that the professor was required to teach only one side of the issue, had to accept industry officials watching him teach, and had little clout to fight back since he didn’t (and still doesn’t) have tenure.”

So it goes. Schools need cash and corporations have it. Would a school bow to its donors? Are schools market immune? Of course they aspire to be but the reality is different. Further as public schools lose the endowment race, they will be more and more beholden to outside funding. I am not, repeat not, saying that schools should operate so that they bow to corporate requests. I am saying that the issue is alive and well and not so easy to combat. If the allegations are true, Hunter seems to be the easy case, don’t do it. The harder ones will be the subtle questions of hiring, curriculum, and building funds which can easily look like a decision based on lack of funds when perhaps other interests scuttled the project.

Hat Tip: Slashdot

Image: Manuel Dohmen WikiCommons

License: GNU Free Documentation license, Version 1.2

3

Short courses?

Greetings from (mostly sunny) Champaign-Urbana, where I’m spending the week, teaching a short course on Federalism and the Making of American Corporate Law at the University of Illinois. Under the law school’s short-course program, the brainchild of Ralph Brubaker, my former colleague at Emory and now Associate Dean here at Illinois, anywhere from five to ten professors, judges, and attorneys come to campus each term, to teach a week-long, one-credit course.

I’m told the students generally love the short courses. My own data – consisting of the (fairly high, I think) enrollment of 27 students in the class, and good participation in the first class (yesterday) – would seem to confirm as much. For the visitors, meanwhile, it can be an occasion to try something new, or at least different, and to spend time with academic colleagues they might otherwise only see in passing, in the hallways at AALS. For Illinois, finally, it’s an opportunity to spread good impressions and good will among legal academics, on the bench, and with the bar. (As Charles Tabb – who’s serving as Interim Dean – put it, it’s a great way “to make new friends.”)

At Emory, we have “accelerated courses,” but of a different sort. Visitors, most commonly hailing from overseas, come for four to seven weeks to teach a class or two. Again, students like it, etc. Obviously, though, the longer format engages a completely different set of potential visitors.

Do other schools do anything similar to Illinois? If not, it’s something I suspect might be well-worth considering.

Should Harvard Last Forever?

Set in the year 3172, Samuel Delany’s novel Nova envisions a future where virtually everything has changed…except for the persistent Harvard University, home to the “eccentric, the brilliant, and the very wealthy.” As Harvard’s endowment balloons to $35 billion, Delany’s work looks less like fantasy and more like prophecy. In our day huge sums of money are the “monumentum perennium” Horace once deemed poetry.

But a former aide to Lynne Cheney has been crusading for wealthy universities to spend more of their endowments, and may be winning the battle of public opinion:

Lynne Munson dismisses Harvard University’s new plan to spend $22-million more annually on student aid as “miserly.” She says Yale University could have shown more leadership by pledging to spend at least 5 percent of its endowment each year, rather than the 4.5-percent minimum it announced last week. And she contends that dozens of other wealthy colleges . . . are guilty of “hoarding” because they do not spend enough to help keep tuitions down.

There are lots of tricky issues here, complicated by an increasingly winner-take-all society and the pernicious USNWR rankings rat race. As the compensation of I-bankers and CEOs rises to stratospheric levels, the top universities may feel that they have to keep raising their own salaries to keep pace. Moreover, competitive pressures are going to lead more universities to try to increase their average incoming SAT scores by offering merit scholarships and providing other amenities.

Robert Frank has proposed curbing that kind of “positional competition” with progressive taxes. Will a spending requirement for endowments have similar effects? Republican senator Charles E. Grassley appears to believe that it will at least lead to tuition relief; he has said

requiring wealthy universities to spend a higher percentage of their endowments could help “more working families see the benefits” of lower tuition bills. . . . The Senate proposal, which has yet to be introduced as legislation, could require rich universities to spend at least 5 percent of their endowment, as private foundations are now required to do, or lose the tax exemption they enjoy on their endowment earnings.

Seems like a fair proposal to me–and it also gives wealthy schools ample opportunity to keep open (in some form) to the end of time.