Randomized Controlled Trials in Nonprofit Settings: Gold Standard Administrative Burdens & the Quest for Organizational Legitimacy

Background and Purpose: Three strands of literature -- evidence-based practice, organizational legitimacy, and administrative burden -- are particularly salient to nonprofits’ growing service delivery efforts in addressing poverty and inequality. The proliferation of nonprofit-based randomized controlled trials (RCTs) and their growing influence on state-sponsored programs reflects the coalescence of these dynamics and demonstrates how such endeavors not only influence the sector but also broader policy agendas. Yet, nonprofit programs are typically client-centered, specifically designed to “meet clients where they are at”, which may at times conflict with requisite standardized treatment regimens of RCTs. These complexities suggest that fielding service-based RCTs in nonprofit settings may ratchet up demands via bureaucratic tasks, legitimacy-seeking pressures, and requirements for the acquisition of additional evidence-based expertise that, in turn, may generate new costs for administrators, staff, and clients.

Methods: To explore these dynamics, we draw on a case study from a large southwest nonprofit’s team-based case-management RCT (2015-2018) designed to improve earnings and work and reduce debt among the treatment group. The intervention used two-person case management teams to attenuate frontline workers’ bureaucratic burdens while also providing emergency financial assistance and wrap-around services to the treatment group. We leverage ethnographic (300 hours of agency observations, over 2.5 years) and interview data (administrators and caseworkers, N=42) at two points in time. Our protocol included questions on front-line staff perspectives on administrative burden, client engagement, and participating in an RCT. We coded a priori themes particular to our research protocol. We then re-coded for emergent themes.

Findings: We find client-centered nonprofit RCTs bring distinct organizational administrative costs that affect both workers and clients. These burdens and their related costs are not generated via the state per se but by methodological demands of RCTs and the introduction of potentially perverse organizational legitimacy incentives, which we collectively refer to as gold standard costs.

Conclusion: We find that viewing RCTs as legitimacy-seeking endeavors taken on by both funders and nonprofits provides a helpful framework for understanding how the burdens and costs of investing in an RCT may, in fact, outweigh the benefits for nonprofit human service organizations. Further, illuminating these burdens may help delineate some of the challenges that arise in taking interventions to scale—a frequent goal of RCTs and an increasingly common goal for nonprofits as policy entrepreneurs. While this mode of research has increasingly dominated funding and research, this study helps challenge researchers to consider the implications of RCTs within the context of nonprofit organizations. These insights are especially relevant to researchers and funders who often engage in and endorse RCTs. Finally, this framework aids social workers who work within funding agencies or foundations or those who manage or provide front-line services in nonprofits to better prepare for the potential and unexpected hazards of RCT evaluation on organizational capacity, interests, and client experiences.