Methods: In this paper we use qualitative data from an implementation study of an RCT that administered intensive case management services to the treatment group to reduce poverty and improve work at a large non-profit in the southwest United States. We draw from two waves of in-depth interviews and field observations with non-profit managers and front-line workers. We developed research questions based on existing literature that suggests research can place burden on workers and clients. Our protocol included questions on front-line staff perspectives on administrative burden, client engagement, and participating in an RCT. We coded a priori themes particular to our research protocol. We then re-coded for emergent themes. A major emergent theme across administrators and front-line staff was the importance of the RCT model to organizational legitimacy among donors and the research community, including demonstrating positive effects of the program on client outcomes. Based on our deductive analysis of the data, we illuminate previously unexplored organizational implications that accompany the use of RCTs.
Results: We find that HSO managers often view RCTs as a way to build or maintain organizational legitimacy among key stakeholders—foundations, researchers, and private donors. However, managers often fail to anticipate the costs of RCTs experienced by both workers and clients. These costs include difficulty in maintaining client engagement, worker resistance and discretion, staff turnover, and decreased organizational trust from front-line workers.
Conclusions & Implications: We find that viewing RCTs as legitimacy-seeking endeavors provides a framework for understanding how the burden of investing in an RCT may, in fact, challenge or complicate the benefits. While this mode of research has increasingly dominated funding and research, this study helps challenge researchers to consider the implications of RCTs within an organizational context. These insights are especially relevant to researchers and funders who often engage in and endorse RCTs. Finally, this framework aids social workers who work within funding agencies or foundations, manage HSOs, or provide front-line services to better prepare for the potential impact of RCT evaluation on organizational practices.