Session: Implementing Large Scale Randomized Controlled Trails: Impact On Public Policy and Program Development (Society for Social Work and Research 15th Annual Conference: Emerging Horizons for Social Work Research)

Background: Policy and practice are driven increasingly by research evidence. In recent years, researchers have been playing a greater role in generating evidence through randomized controlled trials (RCTs) in applied settings. This symposium presents three studies that use large-scale RCTs to test innovative social programs. Conducted in public education, child welfare, and asset building, these RCTs provide concrete examples of how social work research can influence public policy and program development. For each project, presenters will describe their research designs, summarize findings, and discuss critical lessons learned.

Methods: In the first study, data on the effectiveness of the Making Choices (MC) program, a social skills training program for elementary schools, were collected from 841 children attending ten rural schools randomized equally to intervention and control conditions. This cluster randomized trial extended previous findings from three control-group studies. Using administrative and survey data, the second study – the Subsidized Guardianship Waiver Demonstration – includes three RCTs with children in foster care in Illinois (n=2,245), Tennessee (n=1,122), and Wisconsin (n=576). Finally, the American Dream Demonstration (ADD) is a ten-year longitudinal study of 1,103 participants randomly assigned to an individual development account (IDA) program or control group. The three studies use a variety of analytic techniques to account for clustering, survey non-response, differential implementation, and selection bias. Presenters will discuss intent-to-treat (ITT) and treatment-on-treated (TOT) analyses.

Results: Extending previous findings, the results from the cluster-randomized trial of MC indicate that the intervention improved the cognitive concentration and prosocial behavior of 4th and 5th grade children. In addition, compared to children in control schools, the study found significant decreases in the relational aggression of intervention children. TOT analyses found positive dosage effects for behavioral outcomes. The findings suggest that social and character development programs provided across the country under the No Child Left Behind Act and state legislation may be effective if they address the social skills of children and the classroom behavior management skills of teachers.

Using ITT estimates, the second study found that children in foster care assigned to the intervention group experienced significantly increased permanency rates in Illinois, Tennessee, and Wisconsin. In addition, results of TOT analysis indicate that the availability of subsidized guardianship as an option significantly reduced lengths of stay in the foster care system. Results of this study played a key role in the inclusion of the Guardianship Assistance Program in federal foster care legislation in 2008. Findings from an ITT analysis of the ADD data indicate that the intervention led to a significant increase in homeownership. In addition, preliminary findings from the recently collected Wave 4 data indicate that this homeownership effect was sustained six years post-intervention. Evidence from the ADD led to passage of the federal Asset for Independence Act and precipitated the adoption of IDAs in the other countries across the globe.

Conclusions: This symposium describes the outcomes of three RCTs. Taken together, these studies demonstrate the potential impact of rigorous social work research on program development and public policy.