Charter Schools and Student Outcomes: What Have We Learned Over Two Decades?
American Institutes for Research/CALDER
Georgia State University/CALDER
CALDER Policy Brief No. 18-0919
Over the past two decades, charter schools have become the most popular form of school choice, especially in urban school districts. As such, a great deal of empirical research has focused on charter schools. Looking at the literature on the student achievement effects of charter school attendance, the weight of the evidence suggests a moderately positive effect with significant heterogeneity in effectiveness across different types of charter schools and across different states/school districts. For example, “No Excuses” charters such as Knowledge is Power Program (KIPP) charter schools have been shown to outperform other charters and traditional public schools in raising student achievement. Educational attainment effects of charter schools have been more positive, with significant effects on high school graduation, college enrollment, and persistence in college. That said, there is still need for more research on (1) the effects of charter schools on later life outcomes including earnings and risky behavior; (2) whether effective charter providers will remain effective at a larger scale; (3) whether the policies and practices of effective charter schools can be successfully implemented in the traditional public school sector; and (4) whether certain state/school district policies better facilitate the growth of an effective charter school sector.
Charter schools are public schools that are granted more autonomy than traditional public schools in exchange for meeting certain conditions outlined in a charter agreement (Gawlik, 2016). These schools have become increasingly popular in urban school districts where the demand for alternative schooling options is typically higher. Figure 1 depicts this trend and presents the change in the number of charter schools and charter school enrollment in the United States over the last two decades. Between 1999 and 2017, the number of charter schools nationwide has soared from less than 100 in 1999 to over 6,000 in 2017 with more than 2.5 million students are enrolled in public charter schools.
Despite this trend, charter schools remain to be a highly debated reform strategy to improve student outcomes in the United States. Charter school proponents argue that these reforms could lead to better student outcomes because they induce competition between schools (and hence serve as the “tide that lifts all boats”) and potentially produce better student-school matches as charter schools have more flexibility in responding to the educational needs of their students. Further, they claim that charter schools level the playing field for disadvantaged students whose families cannot afford private school or homes near better traditional public schools. Opponents, on the other hand, argue that charter schools hinder the progress of low-performing public schools by attracting the ‘best’ students and withholding much needed funds as students depart and enrollment numbers decline.
That said, examining the causal effects of charter schools on student outcomes is difficult due to several empirical challenges. Perhaps the most important issue in this context is that students are not randomly assigned to charter schools: Because charters are schools of choice, students who attend them may tend to have better (or worse) outcomes than those who attend traditional public schools. For example, charter students may be highly motivated or may have parents who place relatively high value on educational quality, and have higher student achievement regardless of their school. Therefore, the observed differences in student outcomes between charter students and students in traditional public schools could be driven by differences in these factors instead of differences in quality of schools.
To deal with this selection issue, the existing literature on charter school effectiveness relies on three empirical strategies. The first strand makes use of “matching” models and compare outcomes for students who attend charters with students who have similar observable characteristics that attend traditional public schools. While these studies can compare a broad range of schools, matching models do not account for differences among students which are not readily measured such as educational motivation and parental involvement.
The second strand relies on student fixed-effects models and compares outcomes for the same student when they attend a charter to the outcomes when they attend a traditional public school. This approach holds constant everything about a student that does not change over time, but there may be time-varying factors, such as divorce or an unusually bad year in school, that cause the switch and affect student outcomes. This could lead to falsely attributing achievement gains to the quality of the charter school, when the student may have “bounced back” even if they had not switched schools.
The third strand makes use of enrollment lotteries in oversubscribed charter schools and compare performance of lottery “winners” who attend charters and “losers” who apply but end up attending traditional public schools. This approach is akin to randomized control trials and arguably has the highest internal validity of the three aforementioned methods. That said, these studies may have limited external validity as their findings are only applicable to charter schools that receive more applications than the number of seats available.
There is an extensive literature examining the effects of charter school attendance on student achievement, yielding mixed results. For example, lottery-based studies in Boston and New York City of “No Excuses” charter schools (e.g., KIPP and Success Academy) find significant benefits on student test scores (e.g., Abdulkadiroglu et al., 2011; Angrist et al., 2013; Dobbie & Fryer, 2011; Hoxby & Murarka, 2009); however, other lottery-based studies (e.g., Furgeson et al., 2012; Gleason et al., 2010) and studies using quasi-experimental methods such as matching and student fixed-effects (e.g., Booker et al., 2007; Davis & Raymond, 2012; Nichols and Özek, 2010; Hanushek et al., 2007; Bifulco & Ladd, 2006; Sass, 2006; Zimmer & Buddin, 2006; Zimmer et al., 2009, 2012) have found more mixed results.
Betts and Tang (2018) provide a meta-analysis of this literature. Table 1 summarizes their findings on the first-year effects of attending a charter school on reading (top panel) and math scores (bottom panel) broken down by grade level. The first three rows in each panel present the average effect sizes based on meta-analysis excluding studies on KIPP charter schools and the fourth row presents the average effect sizes for KIPP middle schools.
Table 1: First-Year Effects of Attending a Charter School on Reading and Math Scores: Average Effect Sizes by School Type and Subject
The results indicate positive average effects of charter school attendance in the first year that are larger in middle school and in math. Specifically, the first column suggests that attending a non-KIPP charter school increases reading achievement by 1.8 to 5.4 percent of the standard deviation in reading and by 3.3 to 9.7 percent in math. In comparison, Chingos (2013) summarizes that the one-year class size reduction effect is a 7 percent of the standard deviation improvement in test scores for a 10-student reduction in class size based on research by Krueger (1999).Notes: Average effect sizes are calculated based on the meta-analysis provided in Betts and Tang (2018). * indicates that the estimate is statistically different from zero at 95% confidence level.
The second and third columns of Table 1 present these charter effects in a different way and examine how a charter school student’s academic ranking changes after one year of charter attendance if the student starts the year at the 50th and the 25th percentile of the test score distribution respectively. Once again, changes in percentiles are larger in middle school and in math, yet represent rather small effects on test scores. For example, the results suggest attending a charter school for one year improves the academic ranking of students at the 50th percentile initially by 0.7 to 2.2 percentiles in reading and 1.3 to 3.9 percentiles in math.
Finally, the last row of each panel in Table 1 highlights the differences in achievement effects between “No Excuses” charter schools and other charter schools. In particular, the average first-year effect of attending a KIPP middle school on reading scores is roughly three times the effect of attending a non-KIPP charter middle school, and almost four times the effect of attending a non-KIPP middle school on math scores.
In contrast to the vast literature about the effects of charter attendance on student test scores, relatively little is known about the effects on educational attainment. This is important because charter schools could affect educational attainment without necessarily having an effect on student test scores, for example, if they have stronger pipelines into college.
We summarize the literature on charter schools and educational attainment that has emerged in the last decade in Table 2. The majority of these studies find positive and significant effects of charter school attendance on high school graduation and postsecondary outcomes. For example, using data from Chicago and Florida, Booker et al. (2011) find that attending a charter high school increases high school graduation by 7.4 to 14.8 percentage points and college enrollment by 8.2 to 10.3 percentage points. Sass et al. (2016) extends this analysis in Florida and finds that charter enrollment leads to a 12 percentage point increase on the likelihood of attending college two years in a row. Angrist et al. (2016) finds a negative effect of charter attendance in Boston on four-year high school graduation rates (despite a significant positive effect on the high school exit exam pass rates), no effect on the likelihood of graduating from high school in five years, and that charter attendance leads to a substantial shift from 2-year to 4-year postsecondary institutions.
Table 2: Effects of Attending a Charter School on Educational Attainment
There are several questions yet to be addressed, especially on the effects of charter school attendance on adolescent risky behavior and adult outcomes. To the best of our knowledge, there are only three studies to date that have examined this question. Dobbie & Fryer (2015) track attendees of Promise Academy in Harlem 6 years after middle school admission lottery, and find that girls who attend Promise Academy charter schools are 12.1 percentage points less likely to become pregnant in their teens while boys are 4.3 percentage points less likely to be incarcerated than those not admitted.
The evidence on the effects of charter school attendance on earnings is mixed. Using a sample of students who attended charter middle schools in Florida, Sass et al. (2016) examine the effects of attending a charter high school on maximum annual earnings 10, 11, and 12 years after 8th grade. They find that earnings are $2,300 higher for students who attended a charter high school compared to observationally equivalent students who attended a traditional public high school. On the other hand, Dobbie & Fryer (forthcoming) compare the earnings of students who attended the same non-charter elementary school, but different middle or high schools. They find that (1) each year of charter school attendance lowers average annual earnings at ages 24-26 by $156 and (2) No-Excuses charter schools increase annual earnings by $173, but the estimate is not statistically different from zero.
There is also need for more research on what might explain the heterogeneity of charter school effectiveness across states/districts as illustrated in Betts and Tang (2018). This variation could be partially explained by the quality of the traditional public schools in different locales. For example, Chabrier, Cohodes, & Oreopoulos (2016) show that the effects of charter attendance decline when the average achievement increases at the traditional public schools which charter applicants would otherwise attend. That said, relatively little is known as to whether charter effectiveness is related to the charter authorizing laws or whether certain state/district policies and practices better facilitate the growth of an effective charter school sector.
There are several policy levers regarding charter schools. The first is to increase the access to effective charter schools by raising the cap on the fraction of funding dedicated to charter schools with proven records. A recent example is the policy change in Massachusetts, which raised its cap for charter providers that are deemed as “proven providers” in 2010. The critical question in this context is whether effective charter providers will remain effective at a larger scale. While the evidence from Boston is promising and shows that replication charter schools generate large achievement gains on par with those produced by their parent campuses (Cohodes, Setren, & Walters 2019), more research is needed to examine the effects of scaling up effective charter schools.
Another charter school related policy lever is to adopt the policies and practices of effective charter schools in the traditional public school sector, which could alleviate some of the concerns regarding the possible adverse effects of charters on low-performing traditional public schools (e.g., Imberman 2011). There is limited evidence in the literature suggesting positive effects of effective charter policies (e.g., in No Excuses charters) on student outcomes in traditional public schools (Fryer 2014), yet more research is needed to better understand the extent to which these practices can be implemented successfully in traditional public schools at a larger scale.
Abdulkadiroğlu, A., Angrist, J. D., Dynarski, S. M., Kane, T. J., & Pathak, P. A. (2011).Accountability and flexibility in public schools: Evidence from Boston’s charters and pilots. The Quarterly Journal of Economics, 126(2), 699-748.
Angrist, J. D., Cohodes, S. R., Dynarski, S. M., Pathak, P. A., & Walters, C. R. (2016). Standand Deliver: Effects of Boston’s Charter High Schools on College Preparation, Entry, and Choice. Journal of Labor Economics 34 (2):275–318.
Angrist, Joshua D., Parag A. Pathak, and Christopher R. Walters. 2013. “Explaining Charter School Effectiveness.” American Economic Journal: Applied Economics 5: 1–27.
Betts, J.R. & Tang, Y.E. (2018). A Meta-Analysis of the Effect of Charter Schools on Student Achievement. San Diego Education Research Alliance at UC San Diego (SanDERA) Discussion Paper 2018-1.
Bifulco, R., & Ladd, H. F. (2006). The impacts of charter schools on student achievement: Evidence from North Carolina. Education, 1(1), 50-90.
Booker, K., Sass, T., Gill, B., & Zimmer, R. (2011). The Effects of Charter High Schools on Educational Attainment. Journal of Labor Economics, 29(2), 377–415. https://doi.org/10.1086/658089.
Booker, K., Gilpatric, S., Gronberg, T., & Jansen, D. (2007). “The Impact of Charter School Attendance on Student Performance.” Journal of Public Economics 91: 5–6.
Chabrier, Julia, Sarah R. Cohodes, and Philip Oreopoulos. 2016. “What Can We Learn from Charter School Lotteries?” Journal of Economic Perspectives 30 (5): 57–84.
Cohodes, S., Setren, E., & Walters, C. (2019). Can Successful Schools Replicate? Scaling Up Boston’s Charter School Sector. IDEAS Working Paper Series from RePEc. Retrieved from http://search.proquest.com/docview/2223064627/.
Davis, D. H., & Raymond, M. E. (2012). Choices for studying choice: Assessing charter school effectiveness using two quasi-experimental methods. Economics of Education Review, 31(2), 225-236.
Dobbie W. & Fryer R. (forthcoming). Charter Schools and Labor Market Outcomes. Journal of Labor Economics.
Dobbie, W., & Fryer, R. (2015). The Medium-Term Impacts of High-Achieving Charter Schools. Journal of Political Economy, 123(5), 985–1037. https://doi.org/10.1086/682718
Dobbie, W., & Fryer, R. (2011). “Are High-Quality Schools Enough to Increase Achievement among the Poor? Evidence from the Harlem Children’s Zone.” American Economic Journal: Applied Economics 3 (3): 158–87.
Furgeson, Joshua, Brian Gill, Joshua Haimson, Alexandra Killewald, Moira McCullough, Ira Nichols-Barrer, Bing-ru Teh, et al. 2012. “Charter-School Management Organizations: Diverse Strategies and Diverse Student Impacts.” Princeton, NJ & Seattle, WA: Mathematica Policy Research and Center for Reinventing Public Education.
Gawlik, M. A. (2016). The U.S. Charter School Landscape: Extant Literature, Gaps in Research, and Implications for the U.S. Educational System. Global Education Review, 3(2), 50-83.
Gleason, Philip, Melissa Clark, Christina Clark Tuttle, and Emily Dwoyer. 2010. “‘The Evaluation of Charter School Impacts: Final Report.’ NCEE 2010-4029.” Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
Hanushek, E. A., Kain, J. F., Rivkin, S. G., & Branch, G. F. (2007). Charter school quality and parental decision making with school choice. Journal of public economics, 91(5), 823-848.
Imberman, S. (2011). The effect of charter schools on achievement and behavior of public school students. Journal of Public Economics, 95(7), 850–863.
Nichols, Austin, and Umut Özek. 2010. “Public School Choice and Student Achievement in the District of Columbia.” Working Paper 53. Washington, DC: National Center for Analysis of Longitudinal Data. http://www.edweek.org/media/publicschoolchoiceandstudentachievement-calder.pdf.
Place, K. & Gleason, P. (2019). Do Charter Middle Schools Improve Students’ College Outcomes?. NCEE Evaluation Brief 2019-4005.
Sass, T., Zimmer, R., Gill, B., & Booker, T. (2016). Charter High Schools’ Effects on Long‐Term Attainment and Earnings. Journal of Policy Analysis and Management, 35(3), 683–706.
Sass, T. R. (2006). Charter schools and student achievement in Florida. Education, 1(1), 91-122.
Zimmer, R., & Buddin, R. (2006). Charter school performance in two large urban districts. Journal of Urban Economics, 60(2), 307-326.
Zimmer, R., Gill, B., Booker, K., Lavertu, S., & Witte, J. (2012). Examining charter student achievement effects across seven states. Economics of Education Review, 31(2), 213-224.
Zimmer, R., Gill, B., Booker, K., Lavertu, S., Sass, T. R., & Witte, J. (2009). Charter schools in eight states: Effects on achievement, attainment, integration, and competition (Vol. 869). Rand Corporation.
 See, for example, a recent Brookings blog piece on some of the arguments raised by opponents and proponents of charter schools by Paul Hill: https://www.brookings.edu/blog/brown-center-chalkboard/2019/06/07/charter-schools-good-or-bad-for-students-in-district-schools/, accessed on 7/18/2019.