Advancing Measurement of Clinical Supervision to Support the Implementation of Evidence-Informed Practices

Background and Purpose:

Implementation strategies are needed to improve equitable access to care that is evidence-informed and anti-oppressive in order to address disparities in the quality and outcomes of behavioral health services. Clinical supervision is an attractive entry point for supporting implementation efforts because it is nearly universally required for providers’ licensure and quality of care management. Evidence suggests that when supervisors use specific strategies during supervision, providers’ use of evidence-informed practice (EIP) improves. However, the lack of valid measures to assess clinical supervision remains a key barrier to generating effective supervision-focused implementation strategies. The few measures that are available have mixed psychometric strength and are not pragmatic for widespread use. This study developed and evaluated the Evidence-Informed Clinical Supervision Strategies scale (EICSS)—a pragmatic, theoretically grounded measure of the two supervision strategies most strongly linked to improved use of EIP: supervisors’ use of active learning techniques (i.e., modeling, behavioral rehearsal), and supervisors’ provision of data-informed feedback.

Methods:

The reliability and factor validity, convergent validity, and discriminant validity of scores on the EICSS were assessed at baseline in a randomized controlled trial of measurement-based care implementation. The sample included 155 therapists in 21 outpatient behavioral health clinics across 3 states. Confirmatory factor analyses tested the EICSS’s factor structure. Mixed effects analyses tested associations between scores on the EICSS and supervision availability, format, and content. Item characteristics were assessed via multidimensional item response theory analyses.

Results:

The hypothesized factor structure was confirmed (χ²=7.70, df=4, p=.103, CFI=.993, RMSEA=.077, average standardized factor loading = .80). Coefficient alpha was acceptable for both subscales (feedback, α=.73; active teaching, α=.76), which were correlated (r=.60, p<.001). Robust associations between scores on the EICSS and clinically-focused supervision content (r=.37, p<.01) provided evidence of convergent validity. Smaller associations with supervisors’ availability (r=.20, p<.05) and non-significant associations with supervision format (r=.05, p > .05) and clinician demographics provided evidence of discriminant validity. Multidimensional item response theory analyses indicated items are sensitive to detecting a range of supervisors’ use of evidence-informed strategies, from relatively little use (MDIFF = -1.00) to very high use (MDIFF = 2.16). Consistent with expectations, the ‘easiest’ item (i.e., most likely to elicit a positive response) was supervisor modeling; the most ‘difficult’ item was provision of feedback based on supervisor observation of sessions.

Conclusions and Implications:

Centering racial equity in behavioral health services is a critical priority for the social work field. Pragmatic measures will provide actionable insights into supervision strategies that can yield the change necessary to advance anti-oppressive, evidence-informed service delivery in real world settings. The EICSS is a promising and pragmatic measure of behavioral health supervisors’ adherence to two essential ingredients of effective clinical supervision that promote these goals.