![[ Visit Client Website ]](images/banner.gif)
The quality of service implementation in youth development programs is associated with youth outcomes (Durlak & DuPre, 2008). In youth mentoring, programs employing a greater number of empirically supported practices achieve more positive effects for youth participants (DuBois et al., 2002). Accordingly, affiliates of MENTOR/The National Mentoring Partnership created the Quality Mentoring System (QMS) intervention to promote successful adherence to practice standards outlined in the Elements of Effective Practice in Mentoring. Incorporating the strategy of quality rating and improvement systems used in fields such as childcare (Zellman & Perlman, 2008), the QMS intervention begins with assessment of agencies for implementation of recommended practices, followed with assistance in developing a tailored workplan to achieve higher standards, and finally provision of training and technical assistance to help agencies accomplish their improvement goals. A randomized trial was conducted to test the hypothesis that agencies guided through this QMS process would demonstrate improvements in their operations and services.
Methods
Agencies providing formal youth mentoring programs in six states (n=62) were recruited into the study and randomly assigned to receive the QMS intervention or to the waitlist control condition. A survey was administered to an agency leader (e.g., CEO, ED, program director) at baseline (n=53, 85% response) and again 15 months after random assignment (n=47, 76% response). The agency survey assessed organizational practices, indicators of successful organizational functioning, and multiple dimensions of organizational capacity (Bryan, 2011). Intervention effects were evaluated through intent-to-treat treatment-control comparisons using regression analyses, controlling for baseline values where applicable. To better understand QMS intervention processes, in-depth qualitative interviews were completed with staff from a subset of 20 programs and analyzed thematically (Braun & Clark, 2006).
Results
At follow-up, respondents representing agencies in the QMS intervention were more likely to report making changes to their mentoring programs or organizational operations (p<.001), but only 41% of the agencies managed to complete their workplans within 15 months. Relative to control group programs, QMS intervention programs reported experiencing greater improvement in quality of program services (p=.039), greater increase in staff capacity (p=.011), and stronger relationships with an external partner (p=.015). In addition, controlling for baseline levels, the QMS intervention had positive effects on organizational financial capacity (p=.032) and on formal program evaluation activities (p=.039). Case studies are presented to illustrate the nature of program improvements attempted and accomplished, along with associated implementation challenges, in one program that completed its workplan and another that faltered along the way.
Conclusions and Implications
This randomized trial provides evidence of meaningful organizational changes and benefits resulting from the QMS intervention. In particular, the QMS intervention contributes to improvement in organizational financial capacity and formal program evaluation practices. The case examples highlight the changes made through this customized intervention and identify factors associated with different levels of implementation. QMS is a promising approach for quality improvement efforts in youth mentoring programs, and MENTOR is undertaking a national initiative to expand access to the QMS intervention. Implications for organizational improvement efforts in related fields are also discussed.