Wednesday, June 06, 2007

If it helped, would it matter? 

There's a new report from the Center on Education Policy that studies student achievement and achievement gaps since the enactment of No Child Left Behind. Its basic findings include:
1. In most states with three or more years of comparable test data, student achievement in reading and math has gone up since 2002, the year NCLB was enacted.

2. There is more evidence of achievement gaps between groups of students narrowing since 2002 than of gaps widening. Still, the magnitude of the gaps is often substantial.

3. In 9 of the 13 states with sufficient data to determine pre- and post-NCLB trends, average yearly gains in test scores were greater after NCLB took effect than before.

4. It is very difficult, if not impossible, to determine the extent to which these trends in test results have occurred because of NCLB. Since 2002, states, school districts, and schools have simultaneously implemented many different but interconnected policies to raise achievement.
Let's suppose this report is right, and we could further show that there is causality from passage of NCLB to the improvement in student achievement (in a way that statisticians would agree meets the criteria for testing causality.) Would that be enough to convince you that the federal government should have a bigger role in education, or would you continue to argue for local control? In other words, are you philosophically opposed, or is your argument consequentialist (i.e., "federal control doesn't work")? I have an answer to that question myself, but I simply invite yours in comments and I'll put mine in there after a few others are offered.

(h/t: Joanne Jacobs)

UPDATE: Read this over lunch:
In the summer of 2004, the Charlotte-Mecklenburg School District (CMS) in North Carolina determined that ten elementary schools and six middle schools had failed to make adequate yearly progress (AYP) for two consecutive years, and would face the first phase of sanctions under the federal No Child Left Behind Act (NCLB). By law, parents of children in those schools had to be allowed the opportunity to send their children to different schools.
CMS had school choice, but the forms were really difficult to read and get through. Under NCLB, simpler reports were mandated and provided. Parents could choose to get into better schools, but had to go through a lottery to get scarce slots.
Justine Hastings and Jeffrey Weinstein use this natural experiment to explore whether parents changed their school choices in response to the new information about academic performance, and whether allowing them to make a more informed choice led to academic gains. The authors were given access to the district's administrative records. As a result, their sample included information on the less informed school choice made in the spring of 2004 and the more fully informed school choice made in July of 2004. They also had information about school assignment procedures, attendance records, test scores, and student demographics. Of the 6,695 students in their sample, 1,092 students both filled out a new choice form in July and chose to attend a different school.

The authors find that the simplified NCLB notification doubled the fraction of parents choosing a different school, and those parents chose schools with strikingly higher academic achievement. Approximately 16 percent of "parents who received notification responded by choosing schools with test scores that were an average of 1 standard deviation higher than the school that they had chosen to attend just a few months earlier."

A key determinant of whether a parent chose to opt out of a failing school was the existence of higher quality alternatives nearby. Higher test scores at nearby schools significantly increased both the probability that a parent would choose another school, and the test score at the school chosen. The authors find evidence that winning admission to a chosen school reduced serious suspension rates, and that students who gained admission to schools with test scores substantially above their failing school experienced significant improvements in test scores.