Abstract
This paper explores the effect of accountability laws under No Child Left Behind Act (NCLB) on obesity rates among school-aged children in the United States. Our results show that pressures due to school closures for poor performance, rewards for good performance, and assistance to schools that lag behind lead to lower levels of vigorous physical activity. This effect is significant for high school children only. We find no significant impact of school accountability laws on children in grades 3 through 8 after state characteristics such as state obesity rate are taken into account. We also find that state physical education mandates increase physical activity for children in grades 3 through 8 and mitigate the negative effect of accountability pressures on physical activity at the high school level where accountability pressures are most effective at decreasing physical activity and increasing obesity. The study shows that physical education mandates play an important role in promoting physical activity for all grades in our sample.
1. Introduction
Since 1990s, many states in the USA sought to address the problem of public school quality by requiring standardized testing. School accountability became federal law with the passage of No Child Left Behind Act of 2001 (NCLB). Under NCLB, every state assesses schools based on the fraction of students that meet proficiency standards on state curriculum-based examinations in reading and math; these tests are taken by every student annually in grades 3–8 and once in high school. In addition, NCLB mandates that schools publish their scores and states identify poorly performing schools based on students’ adequate yearly progress (AYP). Many states developed harsh penalties for schools that failed to show AYP, including school audit, school reconstitution, and school closures. Although school report cards and ratings of schools based on student performance are now present in all states, the extent of sanctions and assistance for schools that lag behind as well as rewards for top schools varies greatly from state to state.
Schools and educators have been found to change their behaviors accordingly. Previous research shows that accountability pressures of NCLB had both desirable consequences as well as unintended negative effects on school behavior. Although early research into the effects of state accountability laws shows improvements in academic achievement [1–3], concerns have been raised about unintended effects of such laws. With incentives to raise only basic skills, schools may reallocate resources away from art, creative writing, social studies, and physical education [4, 5]. Since health outcomes are not the visible strands of the law, schools have little incentive to emphasize both physical education (PE) and after-class extracurricular activities [5–7]. In view of the fact that only the test scores on standardized tests are measured and rewarded [8] shows that schools focus most of their attention on rewarded goals, to the detriment of other goals, such as health improvement.
Academic pressures brought about by NCLB can decrease physical activity (PA) levels of school-aged children through several pathways. First, accountability pressures may induce educators to prioritize subjects that are being tested under NCLB and cut back on recess, PE, and other extracurricular sports programs. If this change leads to a reduction in overall PA levels, then accountability pressures may contribute to increasing obesity levels among school-aged children. Note that accountability pressures may affect students who are at risk of overweight and overweight the most, as researchers found a relationship between unhealthy weight and poor academic performance [9]. Second, faced with academic pressures, students themselves may allocate more time to academic study and less time to physical activity in school and after school. This effect may be especially strong in states that require an exit test for graduation, thus holding students accountable for academic performance. Lastly, faced with financial pressures, schools are compelled to raise funds through unhealthy food and beverage contracts that contribute to higher BMI [7]. Since poorly performing schools may see loss of funds in some states or may struggle to raise additional funds to improve instruction to meet AYP, this is one of the pathways through which accountability standards may increase obesity among school-aged children.
This paper investigates the relationship between academic pressures created by school accountability and youth obesity, which is a serious public health concern in the US. A survey by the National Center for Health Statistics reports that the number of adolescents who are overweight has tripled since 1980 and the prevalence among younger children has more than doubled. In the 1999–2002 National Health and Nutrition Examination survey (NHANES), 16 percent of children aged 6–19 years are overweight [10]. Not only have the rates of overweight increased, but the heaviest children in a recent NHANES were heavier than those in previous years. Increase in childhood obesity draws intensive public attention since obesity in childhood is correlated with type II diabetes and risk factors for heart disease (such as higher cholesterol and blood pressure) and leads to a growing amount of medical expenses on obesity-related diseases. [11–14] Moreover, obese youths are likely to become obese adults, the fact which creates extra medical and financial consequences [15, 16]. Lastly, childhood overweight is associated with social and psychological problems such as discrimination, poor self-esteem, and discrimination in labor markets that results in lower wages [17–21].
The main cause of the increase in childhood obesity is straightforward: an excess of caloric intake compared with caloric expenditure. Previous research has shown that school policies can make an important contribution to promote physical activity [22, 23]. More recently, states passed laws that require schools to offer PE courses throughout the school curriculum although the required PE time varies from state to state. Estimates show that PE mandates on average increase minutes spent physically active in PE by 31 minutes per week in high school [24]. However, PE mandates tend to crowd out time on after-class sports and thus tougher PE regulation alone does not promote a more active lifestyle among students. Thus far, very little research exists that focuses on school incentives to offer and promote PE and PA. Early research does show a significant relationship between accountability pressures and obesity. In a nationally representative sample, the presence of school accountability and years of exposure to accountability laws significantly increase students’ BMI [25]. In a sample of children in Arkansas, schools within five points of the AYP in either direction have a higher rate of overweight in future years and this effect increases overtime [26].
This study contributes to the previous literature on several fronts. First, we use a nationally representative sample of children in grades 3 through 8 as well as in high school who are most affected by NCLB provisions. Second, we construct two alternative measures of accountability pressures using Quality Counts data published by Education Week and capture the strength of accountability pressures rather than the mere presence of accountability in post-NCLB era. Since all schools adopted some accountability provisions following the passage of NCLB, our results are especially policy relevant in the current high stakes testing environment. We use the variance in accountability pressures across states after NCLB provisions have been adopted in all states to examine the effect of accountability system on overweight. Finally, we look at other state policies that mitigate the effect of accountability pressures on children’s weight. Specifically, we examine the effect of school accountability pressures on physical activity levels in states with and without Physical Education (PE) mandates.
2. Methods
2.1. Accountability Pressures
An important problem with studying the impacts of accountability in the post-NCLB era is that all states are required to adopt accountability policies for Title 1 schools, and this limits the state-level variation with which to identify impacts. Previous literature has developed two alternative measures of accountability pressures that measure either the presence of consequences or the strength of consequences. The consequential accountability index captures the presence of accountability pressures imposed by states prior to NCLB [2, 3]. A state is labeled as a consequential accountability state if it attached consequences to school performance [2]. Consequential accountability index is a categorical variable that assigns a grade of one to all states with mild as well as strong consequences for poorly performing schools [2]. Since all states were required to adopt some consequences such as ratings after the passage of NCLB, all states effectively became consequential accountability states after provisions of NCLB were phased in. In this paper, we use post-NCLB data, and all states in our sample already used ratings in 2003 and therefore are defined as consequential accountability states. However, after NCLB there is a considerable variation in the strength of consequences based on school ratings that identify well and poorly performing schools [2]. We exploit this variation in the strength of consequences to estimate the effect of school accountability laws on childhood obesity.
We use two alternative measures of accountability pressures that schools face across all states. Quality counts data used by this study defines three specific dimensions of the school accountability laws. They are as follows.(i) Assistance: state assists schools that it names low performing.(ii) Rewards: state provides monetary rewards to successful schools.(iii) Sanctions: state authorized to close/takeover/reconstitute failing schools.These state laws apply to all schools including non-Title I schools. Our accountability index is the sum of the three accountability laws above, and it varies from zero (if a state has not adopted either assistance or rewards or sanctions) to three (for states that adopted all three laws). Previous research shows that the same simple sum index based on Quality Counts data leads to significant across-state variability in school accountability pressures [27].
It is important to note that NCLB requires states to impose sanctions on all Title I failing schools. Title I is a US federal program that provides financial assistance to local schools with high percentages of poor children to support the academic achievement of disadvantaged students. National Center for Education Statistics reports that in 2001-2002 school year 25.4% of students were enrolled in Title I schools. In addition, Title I schools may be more at risk of failing to comply with AYP. If all failing schools that are on the border of achieving AYP already face sanctions after NCLB, we may not see a significant impact of state-level accountability laws. On the other hand, school sanctions were found to have insignificant effect on school behavior relative to other accountability laws perhaps due to limited state funds to enforce such sanctions [27]. Since Title I schools face consequences even without state accountability laws, our accountability index by 2003 should have a stronger effect on non-Title I schools after NCLB provisions were phased in for all states. Although we cannot identify whether students in our sample are enrolled in a Title I school, we do a separate analysis by poverty level to test this hypothesis.
We also use Quality Counts data to construct an alternative index of accountability pressures developed by Carnoy and Loeb (2002). The index coding follows the following rules [1] (p. 311).
States receiving a zero do not test students statewide or do not set any statewide standards for schools or districts. States that require state testing in the elementary and middle grades and the reporting of test results to the state but no school (or district) sanctions or rewards (no or weak external pressure) get a 1. Those states that test at the elementary and middle school levels and have moderate school or district accountability sanctions/rewards or, alternatively, a high school exit test (that sanctions students but pressures schools to improve student performance) get a 2. Those states that test at the lower and middle grades, have moderate accountability repercussions for schools and districts, and require an exit test in high school get a 3. Those that test and place strong pressure on schools or districts to improve student achievement (threat of reconstitution, principal transfer, or loss of students) but do not require a high school exit test get a 4. States receiving a 5 test students in primary and middle grades, strongly sanction and reward schools or districts based on improvement in student test scores, and require a high school minimum competency exit test for graduation.
Thus, school accountability varies from zero (no standards adopted) to 4 depending on the number of accountability measures that the state collects. Schools that adopted student accountability in addition to school accountability receive a grade of 5.
We use standards and accountability data available in Quality Counts to construct the same index for 2003. Carnoy and Loeb (2002) index assigns a higher score to states that put stronger pressure on schools and adopted more severe consequences such as school closures; it also uses student accountability along with school accountability [1]. The index varies from zero to five, and only states in which graduation is contingent upon performance on state-wide exit test or end-of-course exam receive the highest grade of five. Since some states keep students accountable for their performance, such laws may put additional pressure on students to allocate more time to academic study in detriment to physical activity.
Table 1 reports both accountability indexes by state for 2003.
2.2. Physical Activity and Overweight Measures
Physical activity measures are defined as days of active exercise per week. The National Survey of Children’s Health (NSCH) survey asked all respondents: “During the past week, on how many days did child exercise or participate in physical activity for at least 20 minutes that made him/her sweat and breathe hard….” The variable does not measure days of PE but rather general PA levels that include PE as well as out of school exercise. It is important to note that accountability pressures can affect both in school as well as out of school physical activity, and we believe that our PA measure will capture both effects.
In our study, we estimate probability of a child falling into at risk of overweight and overweight categories. We use Body Mass Index (BMI) to identify children who are “at risk of overweight” and “overweight.” BMI is defined as the individual’s body mass divided by the square of their height. Since children are growing, most studies apply percentiles rather than adult BMI to capture obesity measures among children. A child with a BMI greater than the 85th percentile is defined by NSCH data “at risk of overweight” and child with a BMI greater than the 95th percentile for children of the same age and sex is considered “overweight.”
2.3. Empirical Model
In this study, we use variation in accountability pressures across states to capture the effect of NCLB provisions on childhood obesity. First, we estimate the effect of accountability pressures on students’ physical activity levels. Specifically, we regress individual physical activity levels on a set of state accountability and PE mandate laws and other individual and state characteristics:
In (1) subscript denotes a students and is a vector of exogenous characteristics. Exogenous variables include demographic, socioeconomic, and geographic factors. We expect to be positive and significant if schools in fact decrease physical activity levels in response to higher accountability pressures. We expect to be positive (i.e., we expect PE mandates to increase physical activity) although previous studies did not find a significant effect of PE mandates on the overall PA levels [24]. In addition, in our first stage we interact our measures of school accountability pressures with PE mandates. For students in grades 3 through 8, we define PE mandate as a state requiring a specific time that schools need to allocate to PE every year. For high school students, we use credit hours required in each grade. We hypothesize that accountability laws decrease PA levels but will not be as effective at reducing PA in states that adopted PE mandates.
School accountability differences are likely to be exogenous since the implementation of accountability standards is due to the passage of No Child Left Behind Act rather than physical activity levels and health outcomes, such as prevalence of childhood obesity in the state. We estimate (1) separately for Quality Counts index and Carnoy and Loeb (2002) [1] index reported in Table 1.
In (1), is a vector of child (age, gender, US born), family (household income, maternal education, exercise by parents), and state characteristics (per capita income, state childhood obesity rate). Since obesity rates vary considerably by regions in the United States, we control for regional differences as well.
To gage whether higher accountability pressures translate into higher obesity rates, we first estimate a probit model and regress at risk of overweight and overweight variables on the same set of independent variables described in (1). In this model, we do not control for PA levels, and the effect of PE mandates on overweight is only indirect. Since we are interested in interaction term in a nonlinear model, standard errors were adjusted [28].
However, when estimating probability of a child falling into at risk of overweight and overweight categories, it is important to control for child’s PA levels. Thus, we estimate the effect of potential reduction in physical activity levels brought about by school accountability pressures on the probability of a child being overweight and at risk of being overweight. In this model, child’s PA level is likely to be endogenous. In our analysis, we use instrumental variable (IV) technique to investigate whether accountability laws have a causal impact on overweight rates through PA levels. Our empirical model is based on Newey’s two-step estimator [29]. Equation of interest is
In (2), we estimate a probit of a child falling into at risk of overweight and overweight categories separately. = 1 if a child falls into at risk of overweight and overweight categories. We use state PE mandates as our instrumental variable [24]. We assume that PE mandates affect child’s PA levels but affect probability of a child falling into at risk of overweight or overweight category only indirectly through their effect on physical activity. In our sample correlation between state PE mandates and probability of being at risk of overweight and overweight is close to zero (−0.0037 and −0.0019, resp.). However, some states may implement PE mandates in response to higher childhood obesity. In (2), we capture state characteristics (). Thus, we follow previous research and control for percent of obese children in the state (we do not use our sample to estimate state obesity rates but instead we use obesity data as reported by National State Conference of State Legislature by state and year) as well as state per capita income (reported by U.S. Department of Commerce, Bureau of Economic Analysis) to capture state characteristics that may pressure policy makers to pass PE mandates [24]. We believe that after we control state obesity rates and per capita income, state PE mandates affect PA levels of public school students but are uncorrelated with individual students’ overweight status.
Further, to check whether our measures of accountability pressures are capturing the effect of accountability laws on PA levels and the probability of being overweight rather than some other unobserved state factors, we rerun our regressions for private school children only. If our model captures the effect of accountability pressures on a child’s PA level and overweight, we should not see any effect on children in private schools since private schools are not affected by NCLB provisions.
2.4. Data
School accountability data used in this study comes from the annual Quality Counts survey published by Education Week that collects state-level data on school standards and accountability for all fifty states and the District of Columbia. The data is publicly available from http://www.edweek.org/ website.
We collected data on PE mandates using two sources. First, we used the 2001 Shape of the Nation Report (SONR) to determine in which state schools are bound by the PE mandate. In our sample, 76.6% of students in grades 3 through 8 lived in states with PE mandates. We used previous estimates to code PE mandates at the high school level [24]; the paper aggregates SONR (2001) data for high school students, and normalizes it such that 1 credit unit is equal to 1 year of instruction. For high school students credit hours vary from zero to 4, and an average high school student was facing 0.905 credit hours.
We merge data on state accountability laws and PE mandates with youth obesity data collected by The National Survey of Children’s Health (NSCH), a nationally representative individual level data on children aged 0–17 collected in 2003. We further restrict our sample to children attending public schools since state accountability laws apply to public schools only. Children who had missing values for body weight and height and were not in school, home schooled, or attending private schools were excluded. We use NSCH sample weights to produce estimates that can be generalized to the general population.
Table 2 presents descriptive statistics for the entire sample as well as for children in grades 3 through 8 and high school separately. On average, 30.8 percent of the sample students were at risk for overweight, and 15.7 percent were overweight. These numbers are consistent with the latest reports on obesity rates in the United States which find that, among children aged 6 through 19 years in 1999–2002, 31 percent were at risk for overweight or overweight and 16 percent were overweight [30].
Table 2 demonstrates that an average student in our sample lived in a state with 1.6 accountability laws. Vigorous physical activity is defined as students engaged in PA for at least 20 minutes that brought sweat and hard breath. The average number of days of vigorous PA exercise in NSCH is 3.98 days in our sample. Students in grades 3 through 8 are on average more likely to be at risk of overweight and obese than high school students. They also report higher levels of vigorous PA and higher levels of participation in the school lunch program.
3. Results
We first estimate the impact of state accountability pressures and state PE mandates on the general physical activity levels as measured by days per week a child is involved in vigorous exercise of at least 20 minutes. Table 3 presents results separately for children in grades 3 through 8 (columns labeled 1 and 2) and those in high school (columns labeled 3 and 4). Columns 1 and 3 use Quality Counts index while columns 2 and 4 use Carnoy-Loeb index. First, we find that accountability pressures significantly reduce physical activity levels for high school children. We did not find any effect on children in grades 3 through 8. In all regressions, we find that PE requirements are not significant determinants of general physical activity levels in high school. This result is consistent with findings in the previous literature that show that PE laws increase in-school exercise but do not increase general PA levels for high school students [24]. For children in grades 3 through 8 PE mandates tend to increase days of vigorous exercise. Although coefficients are only significant at the 10% level, the magnitude is higher than for other policy variables. At the same time, for high school children whose PA levels are most affected by accountability pressures, PE mandates are actually effective at promoting PA. As accountability pressures increase, for students living in states that passed PE mandates, PA levels are significantly higher relative to children who live in the states without such mandates. Thus, our results show that state PE mandates may be more important than the previous literature that seems to indicate at promoting general PA levels.
We next test whether accountability pressures translate into higher probability of a child falling into at risk of overweight and overweight categories. Table 4 through Table 6 presents results separately for children at risk of overweight (columns labeled 1 and 2) and overweight (columns labeled 3 and 4). Columns 1 and 3 use Quality Counts index while columns 2 and 4 use Carnoy-Loeb index. Table 4 shows that for children in grades 3 through 8 accountability pressures have a positive and statistically significant effect on both at risk of overweight and overweight probabilities and this result holds for both measures of accountability pressures. However, this significant effect of accountability pressures for children in grades 3 through 8 goes away once we control for physical activity levels. At the high school level (Table 5), we only find significant effect on the probability of being overweight. For both grades 3 through 8 and high school we find no effect of PE mandates on overweight measures.
Our IV results in Table 6 show that for high school children accountability pressures significantly increase probability of being overweight but not probability of being at risk of overweight even after we controlled for individual PA levels. This result holds for both of our measures of accountability pressures. For example, an addition of one school accountability law (that increases our quality counts index by 1) increases probability of a child being overweight by 4 percentage points. An increase in Carnoy and Loeb (2002) [1] index by 1 translates into a 2 percentage point increase in probability of being overweight. We find no significant impact of school accountability laws on children in grades 3 through 8 after state characteristics such as state obesity rate are taken into account (results are omitted). It is important to note that accountability indexes are positive and significant determinants of overweight status even after we control for physical activity levels. Thus, accountability pressures decrease physical activity levels and may affect overweight through other pathways such as vending contracts that provide unhealthy food options in public schools.
It is important to note that different state accountability laws (assistance, rewards, and sanctions) may not be equally important. Thus, in addition to using the indexes we entered all three laws separately and found that our results of the effect of accountability pressures are driven by rewards and assistance while sanctions were not significant in all models. This result may be true since in 2003 many states that passed sanctions rarely enforced them. Since all states phased in NCLB requirements by 2003, we may see sanctions playing a greater role since NCLB mandates implementation of sanctions to all schools identified as “in need of improvement”.
Other significant variables include child characteristics: males are more likely to be at risk of overweight than female students but exercise more as well. African-Americans and US-born children are more likely to be at risk for overweight and overweight than whites and foreign-born children, respectively. It is important to note that family’s preferences for healthy living (as captured by maternal and paternal exercise) are more important in increasing PA and decreasing obesity than school and state laws. Also, children from lower income households tend to exercise less and have higher overweight levels. Regional differences are also important: children who live in the West and Midwest are less likely to be overweight.
3.1. Sensitivity Analyses and Falsification Tests
Since state accountability laws may be less effective for Title I schools, we run our model for children above the poverty line. We find that the coefficient on accountability pressures does not change much in magnitude and statistical significance and has the same effect on PA levels for children above 100% of poverty line and above 133% of poverty line. However, we do find that accountability pressures have a greater effect on overweight measures for children above 100% and 133% of poverty line. For example in Table 4 our quality counts index has a significant positive effect at the 10% level on the probability of overweight (with coefficient of 0.0298). When we rerun the same model for children above 100% of poverty line, the coefficient increases in significance and is equal to 0.0361 and is significant at the 5% level (st. error = 0.0160). We see the same trend for a sample of children above 133% of poverty line. Thus, accountability pressures are significant for all children but have a greater effect on overweight levels for non-Title I school children.
The main threat to validity of our empirical model is that states with PE mandates may also have greater prevalence of obesity. We perform several falsification tests to rule-out this possibility. First we found that in our data reports of “father exercises regularly” and “mother exercises regularly” are uncorrelated with state PE mandates. At the middle school level, correlation between PE mandates and paternal regular exercise report is 0.0059 and correlation between PE mandates and maternal regular exercise report is 0.0044. At the high school level, same correlations are 0.0001 and 0.0076, respectively. Regression analysis shows that our variables of regular exercise by parents are not significant predictors of state PE mandates. In addition, we examined the relationship between state obesity rates of US adults (as reported by the CDC) and PE mandates and found no significant relationship.
Finally, we rerun our model for a sample of private school children who are not affected by accountability laws and PE mandates. Congressional mandated “Condition of Education” report shows that private school enrollment is relatively small in the United States (only 6.1 million students in 2001) and is falling over time. About 80 percent of private schools are religious and school population is 73 percent white (versus 55 percent in public schools). As expected, we found no effect of accountability indexes and PE mandates on PA or overweight status of private school children for all grades. This increases our confidence that our model captures the effect of school laws rather than unobservable state characteristics.
4. Implications for School Health
Our paper sheds some light on the effect of school accountability pressures on PA levels in states with and without PE mandates. School accountability pressures tend to have a negative and significant effect on PA levels among high school children and increase probability of a child falling into overweight category. Thus, concerns that increased pressures for school performance are associated with increased overweight levels are justified. This effect is mitigated by state PE mandates. Although PE mandates by themselves do not have a significant effect on general PA levels at the high school level they do increase PA levels in the states with higher school accountability pressures. We also find that PE mandates are more effective at promoting PA for students in grades 3 through 8.
School accountability pressures are not unique to the United States. Leithwood et al. (1999) document school and student accountability pressures in Scotland, Ontario (Canada), The Netherlands, Norway, New Zeeland, Hungary, Germany, and Switzerland [31]. With increasing pressure on schools to prove children achieve academic success combined with publicly available school grade reports, PE and active recess diminish across countries [32, 33]. This trend is unfortunate since the literature review studies do show a positive relationship between PA and academic success [32].
An important limitation of this study is that we only use state level laws. Both PE mandates and school accountability laws may exist on a school district level but are not taken into account in this study. Also, we are unable to distinguish between time spent by children exercising in school and outside of school. Our model only captures the effect on the total time spent on vigorous exercise. More research is necessary to determine the effect of NCLB policies on children’s physical activity levels while in school, nutrition, and other health outcomes.
While school administrators struggle to comply with NCLB provisions and to meet the AYP to avoid harsh consequences, this could occur at the expense of students’ physical health. As schools and students reallocate resources to improve academic achievement, health improvement policies such as PE mandates can be effective at promoting PA and healthier weight. Promotion of physical activity has long been an important component of American school system and should not be left behind in the high stakes testing environment.