Keeping It Real: Helping Funders Develop Realistic Expectations and Measurable Outcomes for Community Change Initiatives

Background and Purpose:

Interest in the potential of community or “place-based” initiatives to address a variety of social ills in largely urban and poor communities across the nation has steadily grown. While community-based initiatives vary widely in their characteristics, all have faced increasing pressure to identify clear outcomes and build their evaluation capacity.  A significant challenge in assessing impact, however, is creating alignment among stakeholders about the types of impact and extent of community change they can expect. Most funders and policy makers aim to change community conditions, and yet budging community-level conditions has proved to be extremely difficult. For instance, a study by the Aspen Institute Roundtable on Community Change (Kubisch et al, 2010) reviewed 48 major community change efforts of the past two decades and found that initiatives rarely produced community-level or “population-level” changes. However, one of the most significant impacts found in almost all the efforts reviewed was improvement in indicators related to civic capacity and social capital.

This presentation provides further evidence for the findings presented in the Aspen report. It reports on the results of a placed-based longitudinal CBPR evaluation of a Foundation’s community building initiative in two urban low-income neighborhoods in Baltimore. It examines changes in neighborhood conditions and resident perceptions, and explores how foundations can develop realistic expectations and effective impact measures for future place-based initiatives.

Methods:

Survey data was collected and analyzed using Independent sample T-tests and ANOVA in two waves from a convenience sample of residents that closely reflected the demographics in each neighborhood (510 in 2013; 349 in 2016) on key outcome variables identified by initiative stakeholders: resident perceptions and actual conditions in safety, cleanliness, greenness, arts/culture, and residents’ levels of different kinds of social capital. Change in residents’ perceptions of neighborhood conditions was compared to key community indicators reflecting actual community statistics.

Results:

Examination of community-level statistics indicates that there were no significant improvements in key neighborhood conditions  (crime, cleanliness, greening, arts/culture) from the beginning of the initiative to the end; in fact, crime increased in both neighborhoods during the initiative period. Survey data indicated that residents' perceptions of neighborhood conditions showed few significant improvements over the study period. The most significant difference when comparing resident surveys from wave 1 to wave 2 is in the area of social capital. In both neighborhoods, mean social capital scores were statistically significantly higher in 2016 compared to 2013. Further, in both neighborhoods respondents who were most involved with the place-based initiative reported significantly higher social capital compared to those least involved. 

Implications:

Overall, the data imply that it may be more realistic to measure changes in social capital rather than real or perceived changes in neighborhood conditions to demonstrate impact in place-based initiatives, offering insight for foundations working with nonprofits and residents in setting expectations and measurable outcomes for community change efforts. Implications of the findings for how funders and nonprofits can develop realistic expectations for future place-based initiatives and identify the most effective impact measures will be discussed.