Private school vouchers and student achievement:Afixed effects quantile regression evaluation ☆Carlos Lamarche ⁎Department of Economics,University of Oklahoma,321Hester Hall,729Elm Avenue,Norman,OK 73019,United StatesAvailable online 3May 2008AbstractFundamental to the recent debate over school choice is the issue of whether voucher programs actually improve students'academic ing newly developed quantile regression approaches,this paper investigates the distribution of achievement gains in the first school voucher program implemented in the US.We find that while high-performing students selected for the Milwaukee Parental Choice program had a positive,convexly increasing gain in mathematics,low-performing students had a nearly linear loss.However,the program seems to prevent low-performing students from having an even bigger loss experienced by students in the public schools.©2008Elsevier B.V .All rights reserved.JEL classification:I21;I28Keywords:School choice;V ouchers;Milwaukee;Fixed effects;Quantile regression“While we celebrate those [students]doing well,we can't turn a blind eye to those who are not ”Rod Paige,US secretary of Education,to the NY Times.Available online at Labour Economics 15(2008)575–590☆Version:March 31,2008.This paper is based on Chapter 2of my dissertation “Quantile Regression for Panel Data ”at the University of Illinois at Urbana-Champaign.I especially thank my advisor Roger Koenker for advice and detailed comments.I am grateful to Dan Bernhardt,Greg Burge,Todd Elder,Lynn Gottschalk,Kevin Hallock,Kangoh Lee,Darren Lubotsky,as well as labor lunch participants at the University of Illinois at Urbana-Champaign and seminar participants at Texas Tech and the 2007Annual Meetings of the European Association of Labour Economists.I thank Professor Cecilia Rouse for providing the Milwaukee Parental Choice program's data.I would like to thank the Guest Editor as well as two anonymous referees for their very helpful comments.⁎Tel.:+14053255857.E-mail address:lamarche@.0927-5371/$-see front matter ©2008Elsevier B.V .All rights reserved.doi:10.1016/beco.2008.04.007576 marche/Labour Economics15(2008)575–5901.IntroductionFundamental to the recent debate over school choice is the issue of whether voucher programs actually improve students'academic achievement,while decreasing inequalities between the best and worst ton Friedman's proposal to use vouchers as a method of improving the quality of education is based on the idea that private schools are more productive than public schools,which is still a highly controversial issue.If private schools are in fact more efficient than public schools,governments can improve the quality of education by offering tuition vouchers to families that want to send their children to private schools.The Milwaukee Parental Choice program,the first program implemented in the US,has been providing vouchers to low-income students to attend private school since1990.The simplicity of the program's idea contrasts sharply with the complexities that plague the program's evaluation.The voucher programs'effect,for example from time t to t',is the difference between what would have happened at time t'if the student was selected and remained in a choice school during the time interval,and what would have happened at time t'if the student was not selected,and remained in the Milwaukee public school.This counterfactual exercise is impossible to obtain using observational data(Rubin,1974).It may be possible,however,to construct groups using a randomized experiment(e.g.,children randomly assigned to attend choice schools and to attend public schools).At time t',the difference between students' academic performances could be attributed to the type of school since the initial assignment was random.However,the Milwaukee Parental Choice program was not implemented under idealized conditions(Witte,2000),and therefore the selection of the control group plays a major role.Given the lack of a valid control group,it is not surprising that previous empirical studies have delivered mixed findings.Rouse's(1998)seminal study used a sample of students in the Milwaukee public schools as a comparison group and individual fixed effects to control for latent characteristics,such as more motivated parents or student abilities that may differ between treatment and comparison groups.The presented approach builds upon Rouse by employing a newly developed fixed effects form of quantile regression that not only controls for unobserved individual heterogeneity,but also allows an examination of the program effects at different points of the educational attainment distribution.The empirical literature(e.g.,Witte,1997;Green et al.,1997;Rouse,1998)has focused upon estimating how the selection to attend the Milwaukee choice schools affects mean test scores. This approach to evaluation may be incomplete for policy analysis of programs serving heterogeneous students.To illustrate,if a stated policy goal is to raise students'achievement to a predetermined minimum standard,it may not be optimal to pursue a program that benefits strong students while causing weaker students to fall further behind.Because education is expected to play an important role in mitigating inequality,“the distribution of achievement gains... constitutes an appropriate criterion for evaluating a school choice intervention”(Howell and Peterson,2002).This paper focuses on the estimation of the selection to attend choice schools on the entire distribution of test scores,considering patterns of achievement in terms of quantiles.There are important reasons why economists,educators,and policy makers are interested in how voucher programs affect students'achievement beyond the mean effect,which is typically estimated with Ordinary Least Squares(OLS)or Instrumental Variables(IV).First,the standard methodology may miss how a policy affects achievement differently at different points of the conditional test score distribution,as illustrated in Eide and Showalter(1998).Second,the possibility that vouchers may increase the differences between high-and low-performing students in the private schools,concern notably reflected in Ascher,Fruchter,and Berne(1996)question“What mechanisms ensure those students [in private schools]who need extra time and attention...receive this more costly instruction?”.Our analysis shows that being selected to participate in the program had an heterogeneous effect on educational attainment,which had not been uncovered by the previous work focusing on mean effects.First,we find that the selected students had an increase in math achievements that ranges from 3.1to 1.2percentile points per year across quantiles.The program seems to dramatically improve the academic achievements of the weak students,having a relatively modest effect on achievements among strong students.Second,although the mean effect suggests that the program has no effect on reading,we find some evidence suggesting that the program improved and reduced the achievements of the low-and high-performing students,ing a new instrumental variable estimator for quantile regression (Chernozhukov and Hansen,2005),we find that the effect of being enrolled in choice schools ranges from 2.3percentile points per year at the 0.1quantile of the conditional distribution of reading scores,to −2.4at the 0.9quantile.This result is important for policy analysis since the Milwaukee vouchers seem to have no effect at the mean,but indeed matter at the tails of the educational attainment distribution.Lastly,our results also reveal that students'gains,measured as the differences between test scores conditional on years since application to the program,are subtle in nature.While the “average ”student in the program had a linear gain in mathematics,high-performing students had a positive,convexly increasing gain,and low-performing students had a nearly linear loss.The offer of vouchers seems to increase the inequalities between low-and high-performing students.However,as we mentioned above,weak students'scores increased dramatically,in levels,compared to public school students'scores.Therefore,the evidence suggests that being selected for the choice program prevented low-performing students from having an even bigger loss experienced by low-performing students in the public schools.The next section briefly introduces a simple behavioral framework,and Section 3presents models and estimators.Section 4describes the data and Section 5the empirical results.Section 6offers conclusions.2.A simple behavioral frameworkWe develop a simple variation of a model of mothers'decisions to enroll children in preschool (Behrman et al.,2004).1The mother of student i maximizes a time separable utility function that depends on consumption C and the child's test score T .There is a technology that produces test scores depending on socioeconomic characteristics x ,attendance to a choice school P ,and stochastic factors εj ,T =T (x ,P ,εj ).We assume that εj is realized after the student attends school.For example,dissatisfaction with teachers,quality of education,school transportation problems associated either with choice school j =1or public school j =0attendance.The variable actual attendance to a choice school P is equal to aS −1,taking value one if the student actually attends a choice school,0otherwise.The variable attendance to a choice school a takes the value one if the mother decides that the student should attend a choice school,and the variable selected to attend choice school in period t −1,S −1,take the value one if the student was stly,each period the mother must spend assets A on consumption and,if she decides to send her child to a choice school,on a fixed cost K .1The framework models heterogeneity,which is unobserved by the econometrician,that can be recovered using quantile regression techniques (Koenker,2005).Alternative strategies are considered in Bitler et al.(2006),Hastings et al.(2006),and Cullen et al.(2005).577marche /Labour Economics 15(2008)575–590It is possible to express the mother's dynamic problem simply as max a {υ0,υ1},where υ0represents the value of not attending a choice school,and υ1denotes the value of attending a choice school.If at time t the student was not selected,her mother trivially chooses a =0because she does not want to give up consumption to pay K .On the other hand,if at time t the student was selected to attend a choice school,the decision is simply,a =1{υ1≥υ0|S −1=1}.This decision depends,among other factors,on the cost of attending a choice school,dissatisfaction with the choice schools,and mother's weight to her child educational achievements.These factors may be different among families,and consequently the probability of attendance may be different as well.3.Evaluation of the Milwaukee voucher programThe previous behavioral framework impose two restrictions on the evaluation of the voucher program,T it ¼T x it ;P it ;e it ðÞ¼x V it A þb P it Àe itð1ÞP it ¼1υ1it z υ0it ÞS it À1¼d þq it S it À1:Èð2ÞThe assumption of linearity in Eqs.(1)and (2)gives a direct link to Rouse's (1998)structural equation model.Rouse estimates a reduced form equation assuming that the effect of selection into the voucher program on the probability of attendance is constant among families (ρit =ρ),yielding an “intention-to-treat ”effect π1that is equal to ρβ.An enormous difficulty in the evaluation of the Milwaukee Parental Choice program is the lack of a valid control group.Initial randomized treatment is needed to obtain an unbiased estimator of the voucher offer,but the program's randomization was based on the applicant group.2Green et al.(1997)used the students not selected to attend choice schools as a comparison group,but this group may lead to selection bias (Rouse,1998;Witte,2000).Witte (1997)used a random sample of students from public schools as a comparison group,but as Rouse (1998)pointed out,there may be differences between the students that were selected to attend choice schools and the students that remained in the public schools.3Only eligible parents who are more interested in their child's achievements apply to the program,which suggests that students who remained in the public system may have parents with a less strong preference for private schools.To get around these problems,Rouse (1998)used both the unsuccessful applicants and a random sample of students from the public schools as comparison groups,and individual fixed effects to control for unobservables such us more motivated parents or student ability.Our approach builds upon Rouse,estimating a fixed effects version of the quantile regression model,Q T it s j j S it À1;x it ;a i ÀÁ¼p 0s j ÀÁþp 1s j ÀÁS it À1þx V it p 2s j ÀÁþa i ;ð3Þwhere Q (·|·)is the τj -th conditional quantile function,and αi is an individual fixed effect.The non-selected students that are omitted for simplicity in Eq.(3)will be introduced in the regression 2The Wisconsin legislature required choice schools to select students at random when the grade was oversubscribed.Although the best schools received more applications,the total number of applications exceeded the number of seats (Witte,2000).3For instance,consider a family making a decision as to whether or not to apply to the program based on υi 0j =x i 0′μ+αi j ,where the binary variable j takes the value 1if the student applies to the program,and the value 0otherwise.From Eq.(2),the participation in the program may depend entirely on parent's weight to her child educational achievements,P i 0=1{υi 01−υi 00≥0}=1{αi 1−αi 0≥0}.578 marche /Labour Economics 15(2008)575–590models estimated in Section 5.If we consider the possibility that the effect of selection to attend choice schools on the probability of attendance ρit is a linear function of ωi (e.g.,individual or family reasons to leave the choice schools)and εit (e.g.,choice program or private school factors),it is straightforward to show that the intention-to-treat effect,p 1s j ÀÁ¼b q ÀF À1e s j ÀÁÀÁ¼Q T it s j j S it À1¼1;x it ;a i ÀÁÀQ T it s j j S it À1¼0;x it ;a i ÀÁ:The model,in its simplest version,assumes that the effect of the voucher offer π1(τj )is decreasing on the quantile τ,and depends on the effect of the selection on the probability of attendance and the differential school effect on achievement.We briefly consider two competing explanations.First,we may consider that the effect is a combination of a treatment effect βand a location-scale shift effect ρ(τj )=ρ−F ε−1(τj ).4We conjecture that while only high-motivated students among low-performing students stay in the choice schools,most students among the high-performing students stay.Since those lower-performing students in the private schools are relatively more motivated than the lower-performing students in the public schools,the ‘intention-to-treat effect ’should be decreasing in terms of quantiles.Alternatively,we can hypothesize that it is a combination of a scale treatment effect β(τj )=−βF ε−1(τj )and a location shift effect β×ρ.It has been argued that private schools may work on all students equally,while public schools that are interested in raising mean achievement may give less amount of attention to lower-performing students rather than higher-performing students (Howell and Peterson,2002,p.156).Although the model now suggests that the voucher program has a different mechanism to affect achievement,we again expect to see a decreasing π1over the quantiles.The empirical evidence will offer below (see,e.g.,Figs.1and 2)provides some support for the latter explanation.3.1.EstimationWe estimate the ‘intention-to-treat ’effect π1(τj )using a quantile regression version of the classical fixed effects estimator introduced by Koenker (2004),b p s j ÀÁÈÉJ j ¼1;b a i f g N i ¼1n o u arg min p ;a X J j ¼1X T t ¼1X N i ¼1x j q s j T it Àp 0s j ÀÁÀp 1s j ÀÁS it À1Àa i ÀÁwhere ρτj (u )=u (τj −I (u ≤0))is the quantile loss function,and ωj is the weight (e.g.,1/J )given to the j th quantile.For simplicity,the vector of independent variables x it ,defined below,is omitted.Parents'and students'unobserved characteristics that may differ between students'selected to attend choice schools and students in the public schools are captured by individual fixed effects αi 's,independent of the quantiles.We advance the method in two directions.First,we propose to use panel-bootstrap to estimate the precision of the estimates of the parameters of the model.Our strategy accommodates to forms of heteroscedasticity replacing pairs {(T i ,S i )}:i =1,…,N }over cross-sectional units i .For small and fixed number of time series observations,the standard errors are consistent provided that the 4It seems natural then to explore the differences between the treatment effect βand the intention-to-treat effect βρat different quantiles of the educational attainment distribution.We first estimated a reduced form model based on Eqs.(1)and (2)to obtain b p 1¼b b q .We then estimated Eq.(1)to obtain βb considering the instrumental variable approach described in Section 3.1.We found that the percentage change between the treatment effect and the intention-to-treat effect has a tendency to increase suggesting that the effect of selection on the probability of attending choice schools is decreasing in τ.579marche /Labour Economics 15(2008)575–590Fig.2.Estimates of the treatment effect.The panels present instrumental variable quantile regression estimates (solid line with dots),classical instrumental variable estimates (dashed line),and .95%confidenceintervals.Fig.1.Estimated effects of being selected for the choice program times years since application on educational attainment.The panels present quantile regression estimates (solid line with dots),classical fixed effects (dashed line),and .95%confidence intervals for the point estimates.580 marche /Labour Economics 15(2008)575–590number of cross-sectional units passes to infinity.Second,we develop a simple framework for testing,described in Appendix A.We also implement an instrumental variable form of quantile regression to estimate the causal effect of the program.By accommodating the function T (·)and the error term,Eqs.(1)and (2)are the system of equations considered by Chernozhukov and Hansen (2005,3.1),who propose an instrumental variable (IV)method for estimating quantile regression treatment effects.We estimate the effect of choice schools on test scores considering,Q T it s j P it ;x it ðÞ¼g s ðÞþb s ðÞP it þx V it A s ðÞð4Þwhere β(τ)is the parameter of interest.The vector x it includes “applicant pool ”dummy variables,a dummy variable for gender,family income,an indicator if family income is missing,and the grade level of the student when the student took the test.Since the selection to attend choice school is exogenous conditional on the school and grade to which the student applied,we use whether the child was randomly selected to attend choice schools S it −1as an instrument for actual enrollment P it .4.DataWe analyze data from the Milwaukee Parental Choice program.5We consider a sample that contains information on applicants to the program including the students that were not selected for the choice schools and students from the Milwaukee public schools as in Rouse's paper (Table 1).There is data on reading and math test scores,based on the normal curve equivalent measure (NCE)from the Iowa Tests Basic Skills (ITBS).The empirical analysis is based on a sample of African-American and Hispanic students who applied to the choice program between 1990and 1993,and a sample of students from the Milwaukee public schools.6Test scores:The Normal Curve Equivalent (NCE)measure is a transformation of the ITBS that produces an integer-level measure,ranging from 1to 99,with a national mean of 50and a standard deviation of 21.We rely on math and reading NCE test scores because (a)the results can be compared with previous paper's findings (e.g.,Witte,1997;Rouse,1998),and (b)the NCE measure has the advantage that it can be averaged.According to test makers,the NCE test score is the same if all the students make one year of progress after one academic year.If some students make more (less)progress in one year than the students in the population of interest,the NCE test score will be higher (lower).75The Milwaukee Parental Choice program was targeted and limited,aimed to provide better educational opportunities to low income families.The program target was poor families living in the city of Milwaukee whose children were not attending private school that year.Families with income,at most,1.75times the national poverty line were eligible to apply (e.g.,the threshold for a family of three was $21,000).6We use Rouse's sample of public school students assuring the comparability of results.As a robustness check,we estimated the effect of the voucher offer considering public school students with income less or equal to 1.75times the national poverty line in 1990.The main empirical findings of this paper are robust to restricting the control group to include only low income students.7One-point increase at the quantile τmay not be translated to the same amount of knowledge in different grades,but addressing this possibility is beyond the scope of this paper.(Note that this same possibility is present when analyzing mean effects).Rather,the present exercise is focused on an investigation of the changes in the educational attainment conditional distribution for the selected students and public school students.Furthermore,we analyze trends over several years to avoid drawing conclusions based on one-year changes that are likely to be noisy measures of performance (Kane and Staiger,2002).581marche /Labour Economics 15(2008)575–590Year since application:This variable is defined as the first year in which a student applied if he either was selected the first time he applied or he was never selected to attend choice school.In case a student applied two or more times,the first year she was accepted is considered the year of application.In the case of the students in the Milwaukee public schools,the year of application was imputed consid-ering year t the “year of application ”for students who have a valid year t +1test score (Rouse,1998).Application lotteries:The probability of being selected to attend choice school is random conditional on the school and grade to which the student applied,because the students were randomly selected when the school was oversubscribed for a particular grade.Consequently,we need controls in model (4)for the school and grade to which the student applied (see,e.g.,Rouse,1998).The application lotteries are time-invariant dummy variables,therefore they will not be included in model (3)because the variables are subsumed in the individual effects.5.ResultsWe shall present the empirical results.Instead of the focus on the mean achievement,we will report the effect of being selected to attend choice schools on the entire distribution of educational attainment.5.1.The heterogeneous impactWe will estimate a somewhat more complicated model than Eq.(3)containing the selected students,the non-selected students,and the sample of students from the public schools.We also introduce interactions between the indicator for whether or not the student was selected to attend a choice school and the years since application because previous researchers have argued that child's educational achievement may not improve immediately.We consider first the simplest version,assuming that gains are equal from year-to-year,T it ¼p 0þp 1d it ÂS it ðÞþp 2d it ÂN it ðÞþX4k ¼À3p 3k d it Àk þp 4S it þp 5N it þa i þe it ð5ÞTable 1Descriptive statistics for the main variables in Rouse's sample analysisVariables StudentsSelectedNot selected MPS Proportion currently enrolled in a choice school0.558(0.497)0.010(0.102)–Math (NCE)scores39.236(18.706)38.108(18.777)40.411(18.483)Reading (NCE)scores37.647(16.265)37.886(17.090)38.700(16.461)Proportion female0.536(0.499)0.470(0.499)0.523(0.500)Family income (in thousands of 1994dollars)12.052(5.915)12.510(5.901)21.750(8.658)Proportion missing income family0.412(0.492)0.542(0.498)0.747(0.435)Proportion African-American0.795(0.404)0.864(0.343)0.868(0.338)Proportion Hispanic0.205(0.404)0.136(0.343)0.132(0.338)Grade level test3.767(2.250) 3.857(2.088)4.337(2.122)Proportion with imputed math test(Rouse,1998methodology)0.074(0.262)0.194(0.396)0.164(0.371)Number of students9863582014Number of observations24628595408MPS stands for Milwaukee public schools.582 marche /Labour Economics 15(2008)575–590where d it −k indicates the number of years pre-or post-application to the program,(d it ×S it )is an interaction between the number of years since application and whether the student was selected to attend choice schools,and Nit is a dummy variable indicating whether the student applied to the program and was not accepted.5.1.1.Fixed effects estimatesWe report estimates for the full sample in Table 2.First,the table shows negative estimates for the effect of being selected into a choice school,suggesting that the selected students had lower math test scores than the students in the public schools when they entered the program.8Although we cannot provide a definitive explanation,it may be possible that program's applicants were not doing well in the public schools.Witte (2000)presents evidence on parental dissatisfaction with their kids'prior school within the Milwaukee public schools finding that (a)choice students seem to have less satisfied parents,and (b)the level of parents'dissatisfaction is statistically different between the choice students and the students in the public schools.He also argues that attitudes of parents toward their kids'prior school may be associated to their children's academic 8Research had indicated that choice students have low test scores relative to low-income public school students (see,e.g.,Witte,2000),but the literature is relatively silent on explaining why the academic achievement of these students was low when they entered the program.As shown in Table 2,Rouse's (1998)estimated mean effect of being selected into the program on math test scores is negative and significant.Table 2Estimates of the selection to participate in the choice program on Math and Reading scores estimates for the full sampleQuantiles0.10.250.50.750.9MeanFixed effects methods Dependent variable=math (NCE)test scoreSelected to attendchoice school−3.649(1.866)−4.270(1.429)−3.321(1.167)−2.288(1.192)−1.173(1.633)−3.391(1.114)Not selected to attendchoice school−2.758(3.192)−5.026(2.109)−3.621(2.170)−2.769(2.150)−1.326(2.707)−3.052(1.741)Selected×numberof years2.323(0.794)3.143(0.608) 2.321(0.454) 1.685(0.477) 1.178(0.669) 2.294(0.399)Not selected×numberof years 0.919(1.643) 1.681(1.122) 1.526(1.138)0.844(0.960)−0.675(1.026)0.672(0.783)Fixed effects methods Dependent variable=reading (NCE)test scoreSelected to attendchoice school−0.320(1.625)−1.390(1.372)0.180(1.254) 1.400(1.169) 2.780(1.412)0.750(1.027)Not selected to attendchoice school−2.140(3.479)−3.200(2.070)−0.555(2.161)1.350(2.008) 2.605(2.410)−0.644(1.584)Selected×numberof years0.320(0.687)0.690(0.630)−0.120(0.541)−0.700(0.472)−1.160(0.635)−0.249(0.368)Not selected×numberof years 0.440(1.555) 1.500(1.069)0.555(0.898)−0.350(0.813)−0.985(1.015)0.247(0.706)Math regressions include a dummy variable for whether the test score was imputed.The total number of observations are 8729(Math)and 8751(Reading).The standard errors (in parenthesis)are obtained after 1000panel-bootstrap replications.583marche /Labour Economics 15(2008)575–590。
