Are After-School Programs Effective? A Systematic Review and Meta-Analysis of After-School Programs
Methods: The present study used systematic review techniques and meta-analytic procedures to assess experimental and quasi-experimental studies of after-school programs in order to analyze the programs’ effect on academic, behavioral, and socio-emotional outcomes for participants relative to non-participants. A comprehensive and systematic search was undertaken to locate randomized (RCT) or quasi-experimental (QED) studies conducted between 1980 and 2012 examining behavioral, socio-emotional, or academic outcomes of after-school program interventions with at-risk elementary or secondary school students. Descriptive analysis was conducted to examine and describe characteristics of included studies. Effect sizes were calculated using the standard mean difference effect size statistic, corrected for small sample size (Hedges’ g). Meta-analysis, assuming random effects models using inverse variance weights, was used to quantitatively synthesize results across studies.
Results: The final sample included 44 studies. The studies were comprised of 22 journal articles, 11 dissertations, 3 government reports, and 8 unpublished non-governmental reports. The interventions were most frequently conducted in a school (31 studies) or community organization (13 studies).The program participants were comprised of 52.5% males, 77.2% eligible for free or reduced lunch, and had a mean age of 11.8 years. African American was the predominant participant race for 45.5% of the programs. Meta-analytic results suggest small, but non-significant effects on emotional-behavioral outcomes (g=0.04, CI, -0.07, 0.14); attendance (g=0.09, CI, -0.35, 0.18); achievement (g=0.17, CI, -0.15, 0.49); and supervision (g=0.07, CI, -0.17, 0.31). Moderate and significant effects were found for socio-emotional outcomes (g=0.54, CI, 0.04, 1.04). Homogeneity analysis revealed the effect size distribution was heterogeneous, indicating significant variance in magnitude of effects across studies. Moderator analysis revealed differences in mean effects on study and intervention characteristics.
Conclusions and Implications: Despite their popularity and promise, after-school programs are falling short to deliver on their potential to impact a myriad of outcomes for at-risk students. While some ASPs demonstrated positive impacts on some outcomes, these effects are not consistently realized across programs or across outcomes within programs. Moderator analysis revealed study and intervention characteristics that moderated effects, thus identifying important differences in methodological and intervention characteristics that can inform practice and research to improve ASPs. In addition, this SR uncovered methodological shortcomings and gaps in the evidence-base. Practice, policy and research implications will be discussed.