Monday, January 10, 2005

The plural of 'anecdote' is not 'data' 

In an article (available for subscribers only to The Chronicle of Higher Ed) on a lecture given by Richard Sander, whose study has been outlined at the Volokh Conspiracy (see the links at the bottom of that post to the rest of the story), there are some really annoying misunderstandings of statistical studies like Prof. Sander's. Two examples suffice. First,

Richard O. Lempert, a law professor at the University of Michigan at Ann Arbor who is one of four authors of a paper challenging the Sander study, called it "effective rhetoric but poor science."

"If I believed all of Rick's results, I would change my position on affirmative action," he said. "Who wants to send more blacks to law school to have them get poor grades or fail the bar?"

He argued that while it is indisputable and troubling that black students who enter law school are less likely to graduate and more likely to fail the bar, that pattern does not result from affirmative action. Financial difficulties and other circumstances may also be significant factors.

Well, if you take time to read the study, you find that Sander did look at family income as a determinant to graduation rates and passing the bar (pp. 439-40), and it has "a marginal but measurable role." Even correcting for that, affirmative action still plays a role. At least Prof. Lempert is trying to write a competing study; we'll see what that finds in time I guess. Perhaps he has a better way of measuring financial difficulties. But the point is that Sander's study already holds family income constant. Prof. Lempert, by the way, is not a newcomer to this study: His results in a 2000 study, showing that black University of Michigan Law School students succeeded as often as white students, was entered in evidence in Grutter. The results of the Lempert, et al., study should be out in May.

The other is just the usual ignorant comments about statistics:

Vernellia R. Randall, a professor of law at the University of Dayton, said she found his conclusions patronizing.

"It's all about statistics -- it's not about people," said Ms. Randall, who is black. "How does he know what's bad for me if he's never even talked to me?"

See the title to this post, Prof. Randall.