First and foremost, the authors attribute charters’ scores to "selection effects." What this means is that because charter schools are attended only by children whose parents have chosen to send them there, a charter population is not selected randomly. Their scores, therefore, might be reflective not of the quality of the schools, but of the students who have chosen to attend them. And why is that important? Because there is good reason to believe that charters are disproportionately selected by parents whose children are doing poorly in traditional public schools. This means charter scores will be lower, as the students that choose them arrive with lower than average skills and abilities, not because charters schools aren’t educating them.

        Evidence of this is more than just anecdotal. Studies in Arizona and by the California State University at Los Angeles have confirmed that charters disproportionately attract very low performing students, and, more importantly, that they add greater value to a child’s education the longer a student is in one. In addition, the Brookings report cites data showing that charters serve a higher proportion of black students (23% vs. 17% nationally) and Hispanic students (18% vs. 15%) than other public schools, groups often left behind in traditional public schools.

        Indeed, in a briefing about the report, Brown Center Director Tom Loveless went beyond suggesting just that charters attract struggling students -- in Texas, he noted, charter schools are required to serve populations composed of at least 75% at-risk students. Expectedly, of all the states studied, Texas exhibited the largest gap between charter and traditional school achievement. So, not only is it likely that low-performing students tend to choose charter schools, in at least one case charters are required by law to serve mainly struggling children. All this data forced the authors themselves to emphasize that clearly "charters attract large numbers of students who are struggling academically in public schools before ever setting foot on a charter school campus."

        Other factors might also depress charter scores. One of the most important of these deals with the fact that many charter schools are new, often in their first or second year of operation. The report points out that new charters typically have lower scores in their first two years of operation. This is not surprising considering that they are working with brand new staffs, administrators, and curricula, and are often searching desperately for permanent facilities. Add to that that they are frequently dealing with students who are themselves very mobile, and a new charter’s task is clearly daunting.

        The Brookings authors state several times that they have no conclusive evidence about charter achievement. More importantly, they identify what would be a useful measure of charter success – an analysis of student gains over time. The current Brookings study offers only a snapshot of student knowledge, providing no indication of the level of achievement at which students started or how much they’ve learned over time. A study that gauges student gains, in contrast, would provide a useful measure of charters’ effectiveness because it would show how quickly charter students learn compared to their peers. This is an especially important consideration if students do indeed tend to come to charters already behind their peers.

        The Mackinac Center on Public Policy, a Michigan think tank, recently conducted just such a study, and what it discovered shows the great progress charters in Michigan are making. According to Mackinac, charter schools are improving scores on the Michigan Educational Assessment Program (MEAP) tests at rates far exceeding other public schools. In fourth grade reading, for instance, Wolverine State charters recorded a 43 percent increase in the number of students meeting state standards between 2000 and 2001. Traditional public schools saw only a 10.1 percent improvement. In fourth grade math the difference was even more startling -- charters outpaced other public schools 28.3 to 0.5 percent. Finally, in seventh grade reading, charters nearly doubled the success rate of their peers. (Link to Mackinac website for more information.)

        Analyses like Mackinac’s provide the kind of data the Brookings authors say they really need. If Michigan is any indication, similar studies in the future will also likely reveal the great success of charter schools.

        The authors of the Brown report came up with two specific public policy recommendations:

1. Compute gain scores. Only by seeing the rate at which charter students learn can we know the value of such schools.

2. States should implement a grace period for holding charters accountable for academic results – a different "accountability clock," they call it -- so that charters are given a fair amount of time to produce learning.

        The main value of the Brookings study is simple. It makes clear that better studies - like the Mackinac Center’s - that measure student gains over time, must be conducted if we want to get a clear picture of how well charter schools are educating students. Unfortunately, that message has been relayed neither loudly nor clearly enough. As a result, charters have not gotten anything close to a fair hearing on how well they educate kids.

# # #

For additional information on the Brookings report as well as other state-by-state studies that offer different findings, check out CER's Monthly Letter, Back to School 2002.

See also: 

• NEW STUDY INCONCLUSIVE AT BEST -- Previous Research Counters Brookings Findings, CER Press Release, September 3, 2002.

• STUDY: CHARTER SCHOOL STUDENTS SCORE LOWER, by Greg Toppo, Associated Press, as printed in the Washington Post, September 3, 2002.