WITHDRAWN: Analytic Approaches for Measuring Sustained Effects in Quasi-Experimental Designs

An important indicator of interventions' effectiveness is the ability to sustain positive outcomes beyond the end of treatment. Multiple evidence-based clearinghouses assign higher levels of evidence to interventions that demonstrate sustained effects at six or twelve months beyond the end of treatment (e.g., Title IV-E Prevention Services Clearinghouse). Several methodological challenges arise when attempting to demonstrate sustained effects in quasi-experimental designs. Methodological challenges are particularly pronounced when the duration of the intervention might vary across program participants, or outcomes at the end of treatment are substantially different between intervention and comparison groups. Several questions arise:

1. How do researchers estimate "end of treatment" for comparison cases who did not receive the intervention?

2. How do analysts account for time when time-to-event outcomes are significantly different between intervention and comparison groups?

3. What could researchers do when the case outcomes are significantly different between intervention and comparison cases in ways that could affect post-treatment outcomes?

This session draws from a quasi-experimental study of in-home services for child welfare involved families as an illustrative example of challenges and potential strategies. Presenters describe methodological and analytic approaches that have been used in previous studies and demonstrate their implications for study outcomes. Session participants will be invited to provide input on strengths and limitations of different approaches, and also offer examples of when different approaches might be more or less appropriate. At the end of the session, participants will have a list of analytic approaches that could be used to measure sustained effects in quasi-experimental designs, as well as the strengths and limitations of different approaches. Presenters will propose decision-making criteria for participants to consider when determining which analytic approach(es) best fit their study.