Validation of the EBP Process Assessment Scale

Background and Purpose: The evidence-based practice (EBP) movement has stimulated a rebirth of optimism about the integration of research into social work practice. The success of this movement, however, hinges largely on the effective dissemination of the EBP process model through training and the willingness of practitioners to adopt this model (Bellamy, Bledsoe, & Traube, 2006; Gambrill, 2006; Mullen, 2006). This study reports on the validation of a scale to evaluate the impact of such trainings and inform future educational efforts. Building upon our prior (published) report that covered the development of the scale and provided some preliminary data supporting its reliability and validity, the current study provides a confirmatory factor analysis as well as more conclusive support for the scale's sensitivity, reliability, and validity based on data from a larger and more diverse sample.

Methods: To assess the scale's internal consistency reliability and criterion and factorial validity social work practitioners and second-year MSW students were surveyed in four areas: Texas, Missouri, New York City and Toronto. Random sampling procedures were used to survey social work practitioners in all areas except New York, where all field instructors from a large university were surveyed. All second-year Master's level social work students were invited to participate in the study at four large schools of social work known for emphasizing EBP in the curriculum. Additional data on the scale's criterion validity and sensitivity were gathered in pretests and posttests of 97 social workers participating in EBP continuing education workshops. Confirmatory factor analysis procedures were used to assess the scale's factorial validity.

Results: The overall scale had excellent internal consistency (alpha=.94), as did the self-efficacy subscale (alpha=.91). The attitudes (alpha=.83), intentions to engage (alpha=.86) and current EBP behaviors (alpha=.86) subscales had good internal consistency, while the feasibility scale was fair (alpha=.63). Criterion validity was established in two ways: 1) There was significant pre to post change, with large effect sizes for the EBP process workshops (t=16.24, p<.001, d=1.72), and 2) the scale and its subscales were significantly correlated with reported prior exposure to EBP through continuing education and university courses (with most coefficients ranging between .25 and .56). The above results regarding pre-post change also supported the sensitivity of the scale. The confirmatory factor analysis goodness of fit indices suggested that the hypothesized five-factor structure or the scale had overall acceptable to good fit after deleting 6 of the 51 items, and that the additional correlation of four theoretically justified error terms significantly improved the model's fit (X²/df=2.45, CFI=.90, RMSEA=.05, SRMR=.06). A second-order CFA also supported an overall scale factor (X²/df=2.51, CFI=.90, RMSEA=.05, SRMR=.06).

Conclusions and Implications: This study provides additional support for the reliability, validity and sensitivity of the EBP Process Assessment Scale. This is the first validated scale developed to assess practitioner views toward the EBP process and evaluate future EBP training and educational efforts.