An Examination of the Sufficiency of Small Qualitative Samples

Background: Qualitative researchers often make estimates about anticipated sample sizes in advance of data collection. These are typically required for human subjects review committees, grant applications, and resource planning purposes.  Once a study is underway or completed, researchers must evaluate whether the sample has been robust enough to address the research aims.  Researchers have recently begun to conduct methodological studies that examine the point at which data saturation occurs.  These findings suggest that under certain study conditions, rich qualitative findings can be discovered with sample sizes as small as 6 individuals (Francis et al., 2010; Guest, Bunce, & Johnson, 2006; Hennink, Kaiser, & Marconi, 2016) and 3 focus groups (Guest, Namey, & McKenna, 2016).  Clarifying the conditions under which small sample sizes yield meaningful findings will minimize unnecessary burden on participants and decrease expenditure of scarce resources.  Considering three distinct qualitative research studies from different substantive areas within social work, we address the following research questions:  1) What minimum sample size is needed to adequately identify codes within the data?  2) To ensure that all larger themes are partially represented by at least one of the codes that comprises that theme?  3) To fully realize the complete dimensionality of all themes by including all assigned codes?  And 4) Are minimum sample sizes needed consistent across different substantive areas of exploration and modes of data collection, specifically individual interviews and focus groups?

Methods:  Data previously collected for three studies were used.  The Engaging Men Study included 27 adults working in some capacity to end violence against women.  The Social Workers’ Perspectives on Practice in Criminal Justice Settings Study included 15 social workers.  In both of these studies, data were collected through individual interviews.  In the Adolescent Bystander Study, 12 focus groups of adolescents were recruited through local schools and an online research panel.  In all studies, data were collected via semi-structured, open-ended questions and transcribed for analysis.  To answer the research questions while also addressing temporal bias, 10 random samples of each size from n=5 through n=10 (n=2 through n=7 for focus groups) were drawn from each project and examined to see what proportion of the codes and larger themes from each original study’s full sample were present within each sub-sample.

Results:  Across all metrics, code and theme representation occurred at similar sample sizes within the three projects.  Significant coverage of codes ranged from a minimum sample size of 6-9, partial theme representation required minimum sample sizes of 4-6, and substantial theme completion necessitated sample sizes of 7-10 cases across the projects.

Implications:  In three substantive areas, using two methodologies frequently used in qualitative research, findings from small sub-samples adequately identified themes and codes in each area of inquiry.  Broadening the methods previously included in similar examinations, the studies we drew upon incorporated in-person and telephone individual interviews and in-person and real-time online focus groups.  Additional cases rounded out or added slight nuance to identified themes, but the vast majority of codes and themes were present in small samples.