A war by any other name

Fact Check Time Again, Mr. Quinn!
The media regularly covers great charter schools, and news stories about low-performing public schools abound. It would be easy to conclude that charter schools are, on average, better than public schools. It would also be wrong.

Most national research shows that on average public schools perform as well as charter schools or better. Some specific studies find benefits of charters, but biases inherent in this research mean that charter schools’ overall impact remains ambiguous. Given the attention state policy-makers have been lavishing on charter schools and the particular focus they receive in the Obama administration’s competitive grants and proposals for reauthorization of the federal Elementary and Secondary Education Act, it is important to separate charter hype from charter reality.

Charter Schools Nationwide Not Better, Maybe Worse, than Public Schools *

Research on charter schools paints a mixed picture. A number of recent national studies have reached the same conclusion: charter schools do not, on average, show greater levels of student achievement, typically measured by standardized test scores, than public schools, and may even perform worse.

The Center for Research on Education Outcomes (CREDO) at Stanford University found in a 2009 report that 17% of charter schools outperformed their public school equivalents, while 37% of charter schools performed worse than regular local schools, and the rest were about the same. A 2010 study by Mathematica Policy Research found that, on average, charter middle schools that held lotteries were neither more nor less successful than regular middle schools in improving student achievement, behavior, or school progress. Among the charter schools considered in the study, more had statistically significant negative effects on student achievement than statistically significant positive effects. These findings are echoed in a number of other studies.

Variations in Charter School Performance

While research tends to show that charters do not, on average, outperform public schools, these studies have found wide variation in charter quality.

Charter school performance may vary geographically. Studies by Caroline Hoxby and by the authors of the CREDO report both found that charter schools in the New York City tended to outperform public schools in the city, for example, while a 2009 study by the RAND Corporation found that charter middle schools appeared to be falling short of public middle schools in Chicago (in reading) and in Texas (in both reading and math).

Charter school research has also found variation based on student demographics and subject matter. A literature review of studies of charter schools concluded that they frequently produced higher test scores in elementary school reading and middle school math compared to public schools, although the effect sizes were small in the latter case, but that they often scored significantly lower in tests of high school reading and math. The Mathematica study found that charter schools serving the largest proportions of low income or low achieving students had positive effects on students’ test scores, particularly in math; conversely, charter schools serving more advantaged or higher-achieving students had negative effects.

Caveats

However, some of these results may be somewhat misleading. In particular, some of the benefits attributed to charter schools may actually be a result of study designs or due to differences in student bodies between charters and regular public schools. Some bases for concern are discussed below.

Selection Bias in Lottery Studies

Lottery studies of charter schools, comparing students who were accepted through the lottery with those who were not, are often described as the “gold standard” of charter research because they ensure equivalent comparison groups. Without the random assignment that the lottery creates, it is hard to know whether any differences in outcomes between charter school students and public school students are due to differences in the schools, or whether they are a result of differences between those who choose to enroll in charter schools and those who do not. However, using lottery studies to evaluate how charter schools compare to public schools creates two other sources of bias.

First, only some charter schools accept students via a lottery, and the schools using a lottery are probably not representative of charter schools as a whole. Charter schools only hold lotteries if they are oversubscribed, meaning that there are more applicants than the school has room to accept. While a school’s popularity does not inherently correspond with its quality, oversubscribed charter schools may be better on average than undersubscribed charter schools.

Moreover, students and/or families will typically want to attend a charter school only if they believe that the charter school offers them a better option than their public school. While specific characteristics of individual charters may make them attractive to students regardless of the quality of local public schools, in general, students who would otherwise be attending lower-quality public schools would be more likely to enter charter school lotteries than those who would otherwise attend the best public schools.

A lottery study, therefore, is likely to be comparing better-than-average charter schools with worse-than-average public schools.

Charter Students Benefit from Peer Effects

Charter schools also likely benefit from enrolling an easier-to-educate group of students than public schools. On average, charter schools enroll fewer English language learners, fewer students with disabilities, and fewer homeless students in comparison with public schools. Some of the highest-performing charter schools also lose many students, most likely their lowest performers, who often return to local public schools.

Some argue that, even if this is a concern in comparison studies, lottery studies correct for these advantages by focusing only on the subset of students who applied to charter schools. However, neither design takes into account the influence of peer effects.

Imagine that two identical students attend two schools that are identical except for one variable – the composition of their student bodies.

Student A attends School A, which has fewer students living in poverty, fewer students with disabilities, fewer English language learners, and fewer homeless students than School B. Every student in School A has a parent or other figure who is invested in the child’s education and had the knowledge and ability to get the child enrolled in a school of his or her choosing. School A kicks out disruptive students, and “counsels out” other low-performers.

Student B, on the other hand, attends School B, which has more students with challenging learning needs. The parents of its students are a mixed bag – some are very engaged in their children’s education, but others may lack the knowledge or time to be involved. When students who couldn’t hack it in School A get kicked or counseled out, they end up in School B.

Should we expect our hypothetical identical students to show the same amount of academic progress over the course of a school year? It is unlikely — the demographics of students’ classmates have a large influence on their own achievement. For example, the parental education and occupation of a student’s schoolmates’ parents has been found to influence a student’s achievement almost as much as the students own family background status.

There are multiple ways in which a student’s peers can impact his or her academic growth. Children directly influence their peers; students whose classmates are more engaged in class early in the school year become more engaged themselves by late in the school year, while students with less-engaged classmates become disaffected, for example. The makeup of a class also affects teachers, who may need to devote extra efforts to disruptive students or those who are farthest behind, limiting their ability to focus their attention on the rest of their students.

Therefore, Student A will benefit from attending school with more privileged classmates than Student B. Furthermore, the effects of peers appear to be greater for lower-ability students than for higher ability students, which might explain why the Mathematica study found benefits of charter schools for lower-achieving students but not for higher-achieving students.

Implications

Given the wide variation in charter performance and the frequency with which national studies find that charters have negative effects, it is clear that charter schools are no panacea for improving education in this country. Policy-makers would be wise to be realistic about what charter schools are actually likely to achieve — such as alternate curricular offerings, for example – and not simply rely on inaccurate hype to conclude that “charterness” somehow inherently equates to a higher-quality education.

* Some charter school proponents claim charters are public schools. However, while most charter schools are publicly funded, they are privately run.

You are entitled to your own opinion, but not your own facts!

LikeLike