Evaluating Community-Based Positive Youth Development Programs: A Scoping Review of Measures of Positive Racial, Ethnic, and Gender Identity, Social-Emotional Development, and Program Social Environments

Introduction: Identifying meaningful and incremental indicators of success for community-based youth development (YD) programs is a challenge, particularly for programs that aim to support youth who are racially, ethnically and gender diverse. This project aims to evaluate the progress of youth across a county wide set of YD programs. We focused on reviewing measures of both promotive and protective factors, to the exclusion of risk factors, to avoid the deficit centered narratives about minoritized and marginalized youth.  In addition to traditional aims of YD programs largely related to social and emotional development (SED), we prioritized measuring Racial, Ethnic and Gender Identity Development (ERGID) as critical to youth development for all youth, but especially for youth of color. Lastly, we explored measures of Enabling Environments (EE), in order to emphasize the program’s role in creating a positive social environment to promote development, but also to buffer the effects of systems of oppression.

Methods: Scoping review strategies were used to identify measures for the ERGID and EE domains using keywords from the literature. Existing reviews of measures were examined for the SED domain (Deighton, 2014; Gokiert 2014; Haggerty, et al., 2011; Halle, et al., 2016; Humphrey et al., 2011; Jenkins, et al., 2014). Our phase 1 criteria for inclusion were: 1) designed for youth ages 10-18, 2) in English, 3) used in past 20 years. Phase 2 inclusion criteria: 1) measures protective or promotive factors, 2) designed for universal use (rather than screening or diagnosis), 3) reported on psychometric properties, 4) alignment to constructs as defined by youth and program staff, and 5) logistically feasible for county wide administration largely administered by grantee providers online.

Results: Phase 1 of our review resulted 37 measures; Phase 2 resulted in 2 measures of race and ethnicity, 1 of gender identity, 10 of SED, and 3 of EE. All ERGID surveys were designed for research rather than program evaluation, and the two included were the only ones applicable to diverse populations of youth from multiple racial and ethnic backgrounds. For race and ethnicity, we found that measures theorized the differences between race and ethnicity in different ways, making the surveys more or less applicable to different racial and ethnic groups. For SED, a few themes emerged: 1) only 2 of the 10 surveys were designed to be used as pre- post-tests (CTC & YDEKC), 2) only one was designed to measure protective factors across different programs (CTC), 3) only one survey explicitly considered cultural differences in its design (CYRM), 4) only one incorporated youth perspectives into the survey development (WCSD-SEC). For EE, we found that no measures explicitly focused on cultural competence or supporting racial or ethnic identity development.

Conclusions: The findings highlight the limitations of existing measures for program evaluation, particularly in community based settings of diverse youth. Additional research is needed to center the experiences of young people of color and from diverse gender identities. Programs’ roles as a buffer for systems of oppression is under-theorized and unmeasured by most measures.