Assessment of Treatment Fidelity in a Randomized Clinical Trial of Adoption Strategies for Hard-to-Place Foster Youth

Background: Treatment fidelity – the implementation of treatment models as intended – is critical to the development, implementation, and evaluation of evidence-based practices. The value of treatment fidelity in social work research has gained traction. However, there is a limited body of research focused on treatment fidelity in the area of adoption and permanency services.

This paper helps fill this gap by presenting treatment fidelity results from the multi-site evaluation of the Nebraska Adoption Project. In brief, this federally funded project sought to improve permanency outcomes for hard-to-place children by integrating intensive child-focused adoptive parent recruitment strategies into the adoption process.

Methods: An RCT design was utilized to assess the effectiveness of the integrated treatment model as compared to “treatment as usual”. A process evaluation, including treatment fidelity, was carried out to inform the outcomes portion of the study.

Inclusion criteria for the project included youth who (at time of referral) were legally free for adoption or had a concurrent plan of adoption, were not in a prospective adoptive placement, were eight years of age or older, had been in out-of-home care for 18 months or longer, had experienced three or more placements while in care, and had one or more DSM-V diagnoses. Upon referral for adoption recruitment services, youth (N=70) were randomly assigned to one of two treatment group agencies or the TAU agency.

Essential components of the treatment model were identified and fidelity tools were created with input from model developers. In-depth case reviews were conducted on a monthly basis to systematically assess adherence to the model. Fidelity results were aggregated to report the percentage of components that were completed as expected each quarter. At the project’s close, fidelity results were analyzed to evaluate each case’s progress with implementation throughout the course of the project. These results were then analyzed to determine impact on outcomes.

Results: Initial analyses comparing the treatment and TAU outcomes revealed little difference. Reflecting on treatment fidelity, both the quarterly and project end results indicated that practitioners struggled to implement several components of the treatment model with high fidelity, particularly those components that were significantly different than components of the TAU model.

Despite these initial findings, further analyses revealed that the more the treatment protocol was adhered to, the more likely youth in the treatment group were likely to experience positive permanency outcomes. Ultimately, analyses revealed that even incomplete implementation of the treatment protocol resulted in better permanency outcomes than youth in the TAU group.

Implications: Without examining treatment fidelity, the effectiveness of an intervention is difficult to discern, complicating decision-making regarding the development and adoption of new practices. As the field continues its pursuit of evidence-based practices, social work researchers are encouraged to examine treatment fidelity in addition to outcomes when evaluating the effectiveness of interventions. In doing so, it is likely that both the quality and quantity of evidence-based practices will increase, including in the practice area of adoption and permanency services where so few evidence-based practices are currently available to practitioners.