Social Survey Sampling Methodology for Policy and Practice Implementation: Results and Lessons from the Philadelphia Economic Equity Project

Background and Purpose: In September 2023, the Philadelphia Economic Equity Project (PEEP) launched its inaugural baseline survey. The PEEP study is designed to understand economic mobility, experiences of economic hardship, and instability in Philadelphia, PA and expands upon other similar efforts. One of PEEP’s primary aims is identifying and implementing local solutions that promote economic stability and growth, therefore a core concern is the inclusion of hard-to-reach populations including high poverty, Black, and Latinx households. An address-based sampling method was selected to ensure recruitment from these populations and to allow for data to be spatially disaggregated and analyzed. This paper discusses the analytical and decision-making process behind establishing our sampling methodology, with a particular interest in collecting data that could inform policy and practice implementation.

Methods: PEEP utilized a stratified sampling design assigned at the zip code level. There were three target populations the PEEP team wanted to ensure were adequately represented: neighborhoods with disproportionate rates of poverty, and neighborhoods with high concentrations of Black, and Latinx residents. Seven strata were composed, with zip codes being assigned as either high or medium poverty, high or medium concentrations of Black, or Latinx residents, and a seventh for zip codes with low poverty, or low concentrations Black, or Latinx residents. The zip codes with high rates of poverty, and high concentrations of Black or Latinx residents were oversampled to protect against anticipated differential rates of response. Strata were later aggregated into “PEEP Neighborhoods” which were constructed after extensive data analysis and consultation with local policy and practice stakeholders. There are 13 PEEP Neighborhoods in total, which reflect recognized areas in the City, and have sufficient sample sizes to support comparison and differentiation across the city. The survey itself was administered via a mixed-mode web-phone methodology, and offered in six languages across formats.

Results: 20,000 personalized one-page study invitations were sent via mail to Philadelphia residents yielding a cohort of 2,438 participants. This surpassed the team’s quantitative and temporal estimates of up to 2,000 responses. Data collection was open from September to the end of December 2023. Response rates reached as high as 17% in some PEEP neighborhoods, and were about 12% citywide.

Conclusions and Implications: We argue and demonstrate that the social work field is well-situated to tailor-make and collect datasets to directly influence local policy and practice. In today’s data-saturated policy landscape, an issue is not just producing and collecting novel information, but developing mechanisms to make said information actionable, an intentional effort that must be considered when developing data collection and dissemination infrastructure. Composed of researchers and policy professionals with experience in social work, education, public health, and governmental and non-governmental leadership, this paper provides a framework and exemplar of how interdisciplinary teams can construct data infrastructure with local needs and supports in mind.