Bridging the Gap between Translational and Outcome Research in Cardiovascular DiseaseView this Special Issue
Evidential Value That Exercise Improves BMI -Score in Overweight and Obese Children and Adolescents
Background. Given the cardiovascular disease (CVD) related importance of understanding the true effects of exercise on adiposity in overweight and obese children and adolescents, this study examined whether there is evidential value to rule out excessive and inappropriate reporting of statistically significant results, a major problem in the published literature, with respect to exercise-induced improvements in BMI -score among overweight and obese children and adolescents. Methods. Using data from a previous meta-analysis of 10 published studies that included 835 overweight and obese children and adolescents, a novel, recently developed approach (-curve) was used to test for evidential value and rule out selective reporting of findings. Chi-squared tests () were used to test for statistical significance with alpha () values <0.05 considered statistically significant. Results. Six of 10 findings (60%) were statistically significant. Statistically significant right-skew to rule out selective reporting was found ( = 38.8, = 0.0001). Conversely, studies neither lacked evidential value ( = 6.8, = 0.87) nor lacked evidential value and were intensely -hacked ( = 4.3, = 0.98). Conclusion. Evidential value results confirm that exercise reduces BMI -score in overweight and obese children and adolescents, an important therapeutic strategy for treating and preventing CVD.
Cardiovascular disease (CVD) is the leading cause of mortality from noncommunicable diseases worldwide, estimated in 2012 at 17.5 million, or 46.2%, of all noncommunicable disease deaths . By 2030, it is estimated that mortality from CVD will increase to 23.3 million . Two of the major risk factors for CVD are overweight and obesity [3, 4], a global and increasing problem among children and adolescents in both developed and developing countries. To illustrate, between 1980 and 2013, the worldwide prevalence of overweight and obesity among children and adolescents in developed countries increased from 16.9% to 23.8% for boys and from 16.2% to 22.6% for girls . For children and adolescents in developing countries, increases ranged from 8.1% to 12.9% for boys and 8.4% to 13.4% for girls . The deleterious effects of overweight and obesity during the childhood and adolescent periods are both immediate and long term. For example, in a population-based sample of United States (US) children and adolescents 5 to 17 years of age, approximately 70% of obese youth had at least one CVD risk factor . From a long-term perspective, overweight and obesity during childhood and adolescence have been shown to track into adulthood , thereby placing this population at an increased risk for CVD and the mortality associated with such [3, 4]. The tracking of obesity into adulthood is especially noteworthy given that up to 80% of obese adolescents are at risk of becoming obese adults .
Therapeutic strategies aimed at reducing the prevalence of overweight and obesity among children and adolescents are important for reducing the lifetime risk of CVD. One such strategy is exercise, a low-cost, nonpharmacological approach that is available to the vast majority of overweight and obese children and adolescents. In a recent meta-analysis that included 835 overweight and obese children and adolescents, a statistically significant, exercise-induced reduction in BMI -score was reported . While these results are encouraging, all of the included studies were published in journals. This is problematic because studies published in journals suffer from an excess of statistically significant findings . Consequently, such findings may not represent the true truth. This excess in statistically significant findings has been shown to be the result of factors that include, but are not necessarily limited to, selective reporting by investigators [11–17]. At all levels of use (research, practice, and policy), it is critically important to understand the true effects of exercise in overweight and obese children and adolescents. However, while guidelines for the assessment of selective reporting and related biases in meta-analysis exist, all have notable limitations and no adjustment methods are recommended . Recently, however, -curve, a new and novel method that can rule out selective reporting and does not require access to nonsignificant findings, has been developed and validated [19, 20]. Thus, given the CVD-related importance of understanding the true effects of exercise on adiposity in overweight and obese children and adolescents, the purpose of this study was to examine whether there is evidential value that exercise improves BMI -score in overweight and obese children and adolescents.
2.1. Data Source
Data for the current study were derived from a recently published aggregate data meta-analysis that has previously been described in detail elsewhere . Briefly, studies were included if they were randomized controlled trials examining the effects of exercise (aerobic, strength training, or both) on BMI -score in overweight and obese children and adolescents . A total of 10 studies representing 835 overweight and obese children and adolescents (456 exercise, 379 control) were included [21–30]. BMI -score was chosen as the primary outcome based on previous research suggesting its greater validity over other types of BMI-related measures . The focus was on a BMI-related measure over other measures of adiposity, for example, fat mass, given that BMI-related measures not only are the most common method for assessing adiposity, but also are used to define overweight and obesity in children, adolescents, and adults. BMI -scores from each study were calculated by subtracting the change outcome difference in the exercise group from the change outcome difference in the control group and weighting by the inverse of the pooled variance. Overall results for BMI -score from each included study were pooled using a random-effects model that incorporates heterogeneity into the analysis. Heterogeneity was assessed using Cochran’s statistic and [32–34].
2.2. Determination of Evidential Value
To determine whether evidential value exists with respect to exercise improving BMI -score in overweight and obese children and adolescents, -curve, a recent and novel approach, was used [19, 20]. The objective of -curve is to test for evidential value in order to eliminate selective reporting as a reason for statistically significant findings. Statistical inference includes (1) studies that contain evidential value (right-skew), (2) studies that lack evidential value (flatter than 33% power), and (3) studies that lack evidential value and were intensely -hacked (left skew). It consists of the distribution of statistically significant values <0.05 for a group of studies, with nonsignificant values >0.05 not included in the analysis. Right-skewed values are indicative of true effects and thus evidential value because they include a greater number of low (s = 0.01) versus high (s = 0.04) statistically significant alpha values. Probability values that are not right-skewed suggest a lack of evidential value while those that are left-skewed are suggestive of -hacking, that is, investigator-suppression of subsets of nonsignificant results.
Testing for evidential value consisted of two steps. First, for each statistically significant value <0.05, the probability of observing a significant value at least as extreme as if the null were true was calculated. This is known as the value ( value of the value) and was calculated by dividing statistically significant probability values from each study by 0.05. For this study, the probability values were derived from -values calculated from the exercise minus control group differences in BMI -score for each study. In order to maintain independence, the results from one study that included more than one intervention group were collapsed so that only one probability value was included for that study . The second step consisted of pooling the values using Fisher’s method . This yields an overall test for skew with degrees of freedom equal to twice the number of values. Thus, a statistically significant test is indicative of a significant right-skewed -curve and thus evidential value that exercise improves BMI -score in overweight and obese children and adolescents. The absence of a statistically significant right-skewed value suggests either a lack of information to make inferences about evidential value or a lack of evidential value. To test for a lack of information, that is, power, the same approach as for right-skew was used except that values were recalculated for expected -curves using a power of 33% and the study’s sample size, accomplished via the use of noncentral distributions. To test for a lack of evidential value suggestive of intense -hacking (left skew), the same approach was used as for testing for evidential value of a real effect, that is, right-skew, except that the values for left skew were calculated as 1 minus the right-skew value. All calculations were robust to outliers, with values winsorized at 0.01 and 0.99. Chi-squared probability values ≤0.05 were considered statistically significant. Data were analyzed using -curve (version 2.0), a free online statistical program available at http://www.p-curve.com/app2/, version 3.0 of Comprehensive Meta-Analysis , and Microsoft Excel 2010 .
3.1. Changes in BMI -Score
Figure 1 shows a forest plot that depicts the overall results for changes in BMI -score, details of which have been previously described . As can be seen, a statistically significant reduction in BMI -score in favor of exercise was observed as well as nonoverlapping 95% confidence intervals. Heterogeneity was statistically significant (, ) but moderate ( = 58.2%, 95% confidence interval = 15.7% to 79.2%). Changes in BMI -score ranged from −0.29 to 0 while overall results were equivalent to a relative reduction of approximately 2%. Six of 10 results (60%) were statistically significant ().
3.2. -Curve Results
Results for evidential value are shown in Table 1 and Figure 2. As can be seen, there was statistically significant right-skew and thus evidential value that exercise reduces BMI -score in overweight and obese children and adolescents. Consistent with this finding are the nonsignificant results for power and left skew.
4.1. Overall Findings
The purpose of this study was to determine whether there is evidential value that exercise improves BMI -score in overweight and obese children and adolescents. The findings indicate that the included studies contain evidential value that exercise improves BMI -score in overweight and obese children and adolescents and provide much-needed reinforcement to previous work on this topic . These results are important given that (1) overweight and obesity are two of the major risk factors for CVD [3, 4], (2) the worldwide prevalence of overweight and obesity in children and adolescents is high , and (3) a need exists to develop therapeutic strategies aimed at reducing CVD. The former notwithstanding, the results could be questioned given that the magnitude of improvement in BMI -score was approximately 2% and -curve does not directly assess for such. However, as previously reported, gross estimates suggest that approximately one million overweight and obese children and adolescents worldwide could reduce their BMI -score by exercising regularly . From the investigators’ perspective, these findings are important.
4.2. Implications for Research and Practice
The results of the current study provide overall results in relation to the effects of exercise on BMI -score in overweight and obese children and adolescents. However, the dose-response effects of exercise were not examined and when previously examined  did not glean any substantive findings. Given the former and as previously suggested , a need exists to examine the dose-response effects of exercise in a representative sample of this population. Until that time, adherence to the World Health Organization (WHO) recommendation of 60 minutes per day of physical activity for children and adolescents appears appropriate and is in concordance with the WHO year 2025 goals of reducing the global prevalence of physical inactivity across all age groups by 10% as well as halting the rise in obesity . Increased participation in exercise among overweight and obese children and adolescents will also likely contribute to the WHO year 2025 goal of reducing mortality from cardiovascular diseases, cancer, diabetes, or chronic respiratory diseases by 25% .
4.3. Strengths and Potential Limitations
The major strength of the current study is the use of a recent and novel approach to address selective reporting of results [19, 20] and thus provide more convincing evidence regarding the true effects of exercise on BMI -score in overweight and obese children and adolescents. This is critically important given the prevalence of self-report bias and subsequent overestimation of treatment effects in the published literature [11–17]. In contrast, one potential limitation is the fact that the current findings were based on 835 overweight and obese children and adolescents nested within 10 studies [21–30]. Consequently, there may have been a lack of precision given that the larger the number of studies as well as number of subjects nested within each study, the greater the precision of -curve results . Another potential limitation is that -curve can fail to detect studies that lack evidential value because it is significantly right-skewed . Furthermore, -curve does not include values >0.05, including those close to 0.05. As a result, values suggestive of no effect, while extremely infrequent in the presence of a genuine effect, are excluded .
The results of the current study provide evidential value that exercise reduces BMI -score in overweight and obese children and adolescents. Given its CVD-related importance, exercise should be recommended as a therapeutic strategy for reducing BMI -score in overweight and obese children adolescents.
The content is solely the responsibility of the authors and does not necessarily represent the official views of the American Heart Association or National Institutes of Health.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Kristi S. Kelley is the coauthor.
The research reported in this paper was supported by the American Heart Association, Great Rivers Affiliate, Grant-in-Aid 12GRNT11670019 (GAK, Principal Investigator), and the National Institute of General Medical Sciences of the National Institutes of Health under Award no. U54GM104942.
World Health Organization, Global Health Estimates: Deaths by Cause, Age, Sex and Country, 2000–2012, WHO, Geneva, Switzerland, 2014.
C. D. Mathers and D. Loncar, “Projections of global mortality and burden of disease from 2002 to 2030,” PLoS Medicine, vol. 3, no. 11, article e442, 2006.View at: Publisher Site | Google Scholar
S. Klein, L. E. Burke, G. A. Bray et al., “Clinical implications of obesity with specific focus on cardiovascular disease: a statement for professionals from the American Heart Association Council on nutrition, physical activity, and metabolism,” Circulation, vol. 110, no. 18, pp. 2952–2967, 2004.View at: Publisher Site | Google Scholar
P. Poirier, T. D. Giles, G. A. Bray et al., “Obesity and cardiovascular disease: pathophysiology, evaluation, and effect of weight loss: an update of the 1997 American Heart Association Scientific Statement on Obesity and Heart Disease from the Obesity Committee of the Council on Nutrition, Physical Activity, and Metabolism,” Circulation, vol. 113, no. 6, pp. 898–918, 2006.View at: Publisher Site | Google Scholar
M. Ng, T. Fleming, M. Robinson et al., “Global, regional, and national prevalence of overweight and obesity in children and adults during 1980–2013: a systematic analysis for the Global Burden of Disease Study 2013,” The Lancet, vol. 384, no. 9945, pp. 766–781, 2014.View at: Publisher Site | Google Scholar
D. S. Freedman, Z. Mei, S. R. Srinivasan, G. S. Berenson, and W. H. Dietz, “Cardiovascular risk factors and excess adiposity among overweight children and adolescents: the Bogalusa Heart Study,” Journal of Pediatrics, vol. 150, no. 1, pp. 12–17, 2007.View at: Publisher Site | Google Scholar
A. S. Singh, C. Mulder, J. W. R. Twisk, W. van Mechelen, and M. J. M. Chinapaw, “Tracking of childhood overweight into adulthood: a systematic review of the literature,” Obesity Reviews, vol. 9, no. 5, pp. 474–488, 2008.View at: Publisher Site | Google Scholar
S. S. Guo and W. C. Chumlea, “Tracking of body mass index in children in relation to overweight in adulthood,” The American Journal of Clinical Nutrition, vol. 70, pp. 145S–148S, 1999.View at: Google Scholar
G. A. Kelley, K. S. Kelley, and R. R. Pate, “Effects of exercise on BMI z-score in overweight and obese children and adolescents: a systematic review with meta-analysis,” BMC Pediatrics, vol. 14, no. 1, article 225, 2014.View at: Publisher Site | Google Scholar
J. P. A. Ioannidis, “Why most discovered true associations are inflated,” Epidemiology, vol. 19, no. 5, pp. 640–648, 2008.View at: Publisher Site | Google Scholar
A.-W. Chan, K. Krleža-Jerić, I. Schmid, and D. G. Altman, “Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research,” Canadian Medical Association Journal, vol. 171, no. 7, pp. 735–740, 2004.View at: Publisher Site | Google Scholar
A.-W. Chan, A. Hróbjartsson, M. T. Haahr, P. C. Gøtzsche, and D. G. Altman, “Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles,” The Journal of the American Medical Association, vol. 291, no. 20, pp. 2457–2465, 2004.View at: Publisher Site | Google Scholar
P. A. Kyzas, K. T. Loizou, and J. P. A. Ioannidis, “Selective reporting biases in cancer prognostic factor studies,” Journal of the National Cancer Institute, vol. 97, no. 14, pp. 1043–1055, 2005.View at: Publisher Site | Google Scholar
K. Dwan, C. Gamble, P. R. Williamson, and J. J. Kirkham, “Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review,” PLoS ONE, vol. 8, no. 7, Article ID e66844, 2013.View at: Publisher Site | Google Scholar
K. Dwan, J. J. Kirkham, P. R. Williamson, and C. Gamble, “Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis,” BMJ Open, vol. 3, no. 6, Article ID e002709, 2013.View at: Publisher Site | Google Scholar
J. J. Kirkham, K. M. Dwan, D. G. Altman et al., “The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews,” British Medical Journal, vol. 340, article c365, 2010.View at: Publisher Site | Google Scholar
N. McGauran, B. Wieseler, J. Kreis, Y.-B. Schüler, H. Kölsch, and T. Kaiser, “Reporting bias in medical research—a narrative review,” Trials, vol. 11, article 37, 2010.View at: Publisher Site | Google Scholar
J. A. C. Sterne, A. J. Sutton, J. P. A. Ioannidis et al., “Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials,” British Medical Journal, vol. 343, no. 7818, Article ID d4002, 2011.View at: Publisher Site | Google Scholar
U. Simonsohn, L. D. Nelson, and J. P. Simmons, “p-Curve and effect size: correcting for publication bias using only significant results,” Perspectives on Psychological Science, vol. 9, no. 6, pp. 666–681, 2014.View at: Publisher Site | Google Scholar
U. Simonsohn, L. D. Nelson, and J. P. Simmons, “P-curve: a key to the file-drawer,” Journal of Experimental Psychology: General, vol. 143, no. 2, pp. 534–547, 2014.View at: Publisher Site | Google Scholar
A. J. Daley, R. J. Copeland, N. P. Wright, A. Roalfe, and J. K. H. Wales, “Exercise therapy as a treatment for psychopathologic conditions in obese and morbidly obese adolescents: a randomized, controlled trial,” Pediatrics, vol. 118, no. 5, pp. 2126–2134, 2006.View at: Publisher Site | Google Scholar
C. L. Davis, N. K. Pollock, J. L. Waller et al., “Exercise dose and diabetes risk in overweight and obese children: a randomized controlled trial,” JAMA: Journal of the American Medical Association, vol. 308, no. 11, pp. 1103–1112, 2012.View at: Publisher Site | Google Scholar
N. J. Farpour-Lambert, Y. Aggoun, L. M. Marchand, X. E. Martin, F. R. Herrmann, and M. Beghetti, “Physical activity reduces systemic blood pressure and improves early markers of atherosclerosis in pre-pubertal obese children,” Journal of the American College of Cardiology, vol. 54, no. 25, pp. 2396–2406, 2009.View at: Publisher Site | Google Scholar
M. Hagströmer, K. Elmberg, S. Mårild, and M. Sjöström, “Participation in organized weekly physical exercise in obese adolescents reduced daily physical activity,” Acta Paediatrica, vol. 98, no. 2, pp. 352–354, 2009.View at: Publisher Site | Google Scholar
A. S. Kelly, R. J. Wetzsteon, D. R. Kaiser, J. Steinberger, A. J. Bank, and D. R. Dengel, “Inflammation, insulin, and endothelial function in overweight children and adolescents: the role of exercise,” The Journal of Pediatrics, vol. 145, no. 6, pp. 731–736, 2004.View at: Publisher Site | Google Scholar
R. Maddison, L. Foley, C. Ni Mhurchu et al., “Effects of active video games on body composition: a randomized controlled trial,” The American Journal of Clinical Nutrition, vol. 94, no. 1, pp. 156–163, 2011.View at: Publisher Site | Google Scholar
A. A. Meyer, G. Kundt, U. Lenschow, P. Schuff-Werner, and W. Kienast, “Improvement of early vascular changes and cardiovascular risk factors in obese children after a six-month exercise program,” Journal of the American College of Cardiology, vol. 48, no. 9, pp. 1865–1870, 2006.View at: Publisher Site | Google Scholar
E. C.-S. Murphy, L. Carson, W. Neal, C. Baylis, D. Donley, and R. Yeater, “Effects of an exercise intervention using Dance Dance Revolution on endothelial function and other risk factors in overweight children,” International Journal of Pediatric Obesity, vol. 4, no. 4, pp. 205–214, 2009.View at: Publisher Site | Google Scholar
G. Q. Shaibi, M. L. Cruz, G. D. C. Ball et al., “Effects of resistance training on insulin sensitivity in overweight Latino adolescent males,” Medicine & Science in Sports & Exercise, vol. 38, no. 7, pp. 1208–1215, 2006.View at: Publisher Site | Google Scholar
D. L. Weintraub, E. C. Tirumalai, K. F. Haydel, M. Fujimoto, J. E. Fulton, and T. N. Robinson, “Team sports for overweight children: the Stanford Sports to Prevent Obesity Randomized Trial (SPORT),” Archives of Pediatrics and Adolescent Medicine, vol. 162, no. 3, pp. 232–237, 2008.View at: Publisher Site | Google Scholar
M. Inokuchi, N. Matsuo, J. I. Takayama, and T. Hasegawa, “BMI z-score is the optimal measure of annual adiposity change in elementary school children,” Annals of Human Biology, vol. 38, no. 6, pp. 747–751, 2011.View at: Publisher Site | Google Scholar
R. Dersimonian and N. Laird, “Meta-analysis in clinical trials,” Controlled Clinical Trials, vol. 7, no. 3, pp. 177–188, 1986.View at: Publisher Site | Google Scholar
W. G. Cochran, “The combination of estimates from different experiments,” Biometrics, vol. 10, no. 1, pp. 101–129, 1954.View at: Publisher Site | Google Scholar
J. P. T. Higgins, S. G. Thompson, J. J. Deeks, and D. G. Altman, “Measuring inconsistency in meta-analyses,” British Medical Journal, vol. 327, no. 7414, pp. 557–560, 2003.View at: Publisher Site | Google Scholar
R. A. Fisher, Statistical Methods for Research Workers, Oliver and Boyd, Edinburgh, Scotland, 1932.
Biostat, Comprehensive Meta-Analysis (3.3), Biostat, Englewood, NJ, USA, 2015.
Microsoft Corporation, Microsoft Excel, Microsoft Corporation, Redmond, Wash, USA, 2010.
World Health Organization, Global Status Report on Noncommunicable Diseases 2014, WHO Press, Geneva, Switzerland, 2014.