Methods: The study assembled a fidelity team to develop a fidelity checklist using the five-step process developed by Feely and Colleagues. These steps include: 1) defining the purpose and scope of the fidelity assessment used, 2) identifying essential components of the fidelity monitoring system, 3) developing the tool, 4) monitoring fidelity, and 5) using the fidelity ratings during analyses.
Results: The final checklist includes 26 items. Key methodological decisions were shaped by the study context, included determining the purpose and scope of the measure and selecting a method of administration. Since the central goal of fidelity checklists is to determine adherence to treatment models, the fidelity team created a checklist that assessed interventionists’ adherence to two different intervention components that varied significantly in terms of degree of structure. Developing a checklist that adequately captured both intervention elements was further complicated by the fact that key intervention elements included both content and process items. The team addressed this complexity by creating a checklist that assessed the presence/absence of key intervention elements using yes/no responses, rather than measuring the quality or quantity of treatment activities. This response format maximized efficiency and ease of completion of the checklist. This methodological decision was also important given that the treatment setting for this study created challenges around the methods of administration for the checklist; prisons ban all recording devices, making direct observation the only method allowed. However, using outside observers could create reactivity among participants due to the small treatment space available in prison and the distrust and suspicion that pervade correctional settings. Therefore, the only method of administration available in this study was for both interventionists to complete the checklist.
Implications: Fidelity measures the extent to which an intervention is delivered as intended. Failure to ensure treatment fidelity impedes our ability to link study outcomes to the intervention. However, the development of fidelity measures is resource-intensive and shaped by the context in which it will be used. The development of fidelity requires balancing the rigor of the tools and the needs of service settings. This study presents information for researchers to use when developing methodologically rigorous fidelity measures that are applicable to real world settings.