Are After-School Programs Effective? A Systematic Review and Meta-Analysis of After-School Programs

Background and Purpose: Public recognition of the potential of after-school programs (ASP) to impact behavioral and academic outcomes with at-risk students has resulted in an influx of funding and rise in ASPs over the past two decades. Research on effects of ASPs, however, has produced a body of ambiguous evidence.  Prior reviews have been limited in their synthesis methods and inclusion criteria; therefore, this review seeks to improve upon prior reviews by employing a comprehensive and systematic review method and more fully assess moderating variables. The specific questions guiding this review were: 1) Do after-school programs positively impact social, emotional, behavioral and/or academic outcomes? and 2) What are the study, intervention, or participant characteristics that account for differential effects of ASPs?

Methods: The present study used systematic review techniques and meta-analytic procedures to assess experimental and quasi-experimental studies of after-school programs in order to analyze the programs’ effect on academic, behavioral, and socio-emotional outcomes for participants relative to non-participants. A comprehensive and systematic search was undertaken to locate randomized (RCT) or quasi-experimental (QED) studies conducted between 1980 and 2012 examining behavioral, socio-emotional, or academic outcomes of after-school program interventions with at-risk elementary or secondary school students.  Descriptive analysis was conducted to examine and describe characteristics of included studies.  Effect sizes were calculated using the standard mean difference effect size statistic, corrected for small sample size (Hedges’ g). Meta-analysis, assuming random effects models using inverse variance weights, was used to quantitatively synthesize results across studies.

Results:  The final sample included 44 studies. The studies were comprised of 22 journal articles, 11 dissertations, 3 government reports, and 8 unpublished non-governmental reports. The interventions were most frequently conducted in a school (31 studies) or community organization (13 studies).The program participants were comprised of 52.5% males, 77.2% eligible for free or reduced lunch, and had a mean age of 11.8 years. African American was the predominant participant race for 45.5% of the programs. Meta-analytic results suggest small, but non-significant effects on emotional-behavioral outcomes (g=0.04, CI, -0.07, 0.14); attendance (g=0.09, CI, -0.35, 0.18); achievement (g=0.17, CI, -0.15, 0.49); and supervision (g=0.07, CI, -0.17, 0.31).  Moderate and significant effects were found for socio-emotional outcomes (g=0.54, CI, 0.04, 1.04).  Homogeneity analysis revealed the effect size distribution was heterogeneous, indicating significant variance in magnitude of effects across studies.  Moderator analysis revealed differences in mean effects on study and intervention characteristics. 

Conclusions and Implications:  Despite their popularity and promise, after-school programs are falling short to deliver on their potential to impact a myriad of outcomes for at-risk students.  While some ASPs demonstrated positive impacts on some outcomes, these effects are not consistently realized across programs or across outcomes within programs.  Moderator analysis revealed study and intervention characteristics that moderated effects, thus identifying important differences in methodological and intervention characteristics that can inform practice and research to improve ASPs.  In addition, this SR uncovered methodological shortcomings and gaps in the evidence-base.  Practice, policy and research implications will be discussed.