Toward Representation in Recovery Measurement: Insights from Qualitative Analyses for Item Development for a Recovery Capital Inventory Tool

Background and Purpose: Although item specification is foundational to measurement development, developers of new measures often neglect to adequately describe procedures for specifying constructs, generating items, and ensuring content and cultural validity. Prior research has established the importance of integrating input from end users in these processes, with attention to demographic diversity to capture varied perspectives on characteristics of the construct and item wording choices. The primary objective of this paper is to address this gap in the measurement development literature by describing the use of qualitative methods to generate and refine items for a new measure of recovery capital—the social, financial, human, and cultural resources that support recovery from substance use problems—with a diverse sample of people in recovery from alcohol problems.

Methods: The research team generated an initial item pool by reviewing the theoretical and empirical literature on recovery capital and related measures. Recovery research experts (n=9) reviewed and expanded the initial pool to 102 items. Next, individuals working in the recovery field and/or self-identifying as in recovery from alcohol problems were recruited online to participate in an individual interview or focus group via Zoom to provide feedback on item salience and clarity. The team used theoretical sampling to recruit with attention to diversity in characteristics including race/ethnicity, socioeconomic status, gender, and recovery experience. To streamline the review process, the research team organized the pool into sub-constructs and selected one item to represent each sub-construct, yielding 53 items for participant review. Interviews and focus groups were recorded and transcribed. Following primary coding, two researchers conducted secondary coding, noting emergent themes regarding item salience and concerns about language clarity. The full research team convened to review the secondary coding and decide upon item elimination, retention, and revision.

Results: Nine recovery service providers and 23 people in recovery completed a focus group or interview. The sample of people in recovery was 44% people of color, 57% cis-female, 9% nonbinary, 52% low-income, and 52% not currently engaged in treatment or self-help groups. Participant feedback directed the researchers to eliminate items seen as problematic or extraneous (e.g., “I desire to ‘fit in’ with people who have not had alcohol or drug problems”), add new items (e.g., “Bias related to my race or gender has made my recovery harder”), and revise items for linguistic and conceptual clarity. This yielded an initial version of the measure with 49 items to be subjected to pilot testing and cognitive interviewing in the following research phase; only about 10% of these items were unchanged from their original formulation.

Conclusions and Implications: The sample included a high proportion of people from groups vastly under-represented in prior alcohol recovery research, including people of color, women, nonbinary individuals, low-income people, and people not engaged in treatment or self-help groups. The extensive revision and refinement of the item pool demonstrates the critical value of qualitative feedback from a diverse sample of people in recovery in developing inclusive and culturally valid recovery measures.