Designing Measurement Strategies for Randomized Trials of Community-Based Interventions: Exploring the Community Youth Development Study

Background and Purpose: Communities that Care (CTC) is a planning used by communities to prevent behavioral problems in youth. This approach to community practice develops and mobilizes human, political, and social capital to adopt, implement, and sustain effective prevention practices. Measuring the impact of CTC requires a multiple component assessment approach; research that is essential to the responsible and innovative practice of macro-level social work. The first community-randomized trial of CTC represents a partnership between the University of Washington, 5 federal research institutes, 7 state government agencies, and 24 communities within these states. Five years of data collection established baseline equivalency of participating communities matched within state prior to randomization. Over the next five years, a coalition-based approach was used in intervention communities to train community members to use data to set priorities, build consensus, develop action plans, implement strategies, monitor progress, and plan for sustainability.

Methods/Results: Five approaches were used to measure the *implementation* of CTC in participating communities. First, ten coalition members participated in telephone interviews annually across five years. The goal was to assess how the coalitions were structured and how they functioned. Second, community prevention coordinators documented the work of the coalition through meeting minutes, taskforce records, and community action plans. Third, community coordinators and technical assistance providers periodically rated the communities against CTC milestones and benchmarks to assess the degree of CTC implementation. Fourth, coalition members regularly recorded the scope and intensity of the delivery of the specific strategies on the community's action plans. Fifth, prevention service providers, school principals, and civic leaders completed telephone interviews, mail and web surveys bi-annually during the intervention to document the type, number, and scope of prevention activities in their communities. In addition, three approaches were used to evaluate the *outcomes* of CTC. First, ten positional community leaders (the mayor, chief of police, school superintendent), and five knowledgeable leaders recruited through snowball sampling techniques, participated in telephone interviews twice during the baseline period and twice during the intervention period. This information was used to compare the extent to which the intervention and control communities were changing over time in regard to their cross-sector collaboration, knowledge, and adoption of effective prevention practices. These leaders also reported on other community events that could shape the community contexts in which prevention activities were occurring. Second, bi-annual cross sectional student self-report surveys were collected of all assenting 6th, 8th, 10th, and 12 grade public school students to assess risk factors, protective factors, and youth problem behaviors. Finally, a longitudinal panel of students who were in 5th grade at the onset of the intervention was surveyed annually to determine incidence and prevalence rates of adolescent health and behavior problems. Each of the quantitative instruments has adequate internal reliability as well as convergent and predictive validity.

Implications: This presentation demonstrates the diversity of measurement strategies used in a randomized control trial of a community level intervention. The presenters will share lessons learned regarding retention, analyzing missing data, and tensions between internal/external validity and university/community directedness.