Validation of the Revised EBP Process Assessment Scale

Background and Purpose: The evidence-based practice (EBP) movement has resulted in widespread optimism within various helping fields regarding the integration of research into social work practice. The success of this movement, however, hinges largely on the systematic and effective dissemination of the EBP process model through training and the willingness of practitioners to adopt this model (Bellamy, Bledsoe, & Traube, 2006; Gambrill, 2006; Mullen, 2006). This study reports on the validation of the shorter, revised EBP Assessment scale, which consists of 37 items instead of 51 items based on prior confirmatory factor analyses and a decision to shorten the scale by removing the intentions subscale (which was highly correlated with the behaviors subscale due to similar wording). Like the earlier version of this scale, the new scale can be used to evaluate the impact of trainings and conduct surveys of practitioners' EBP views and behaviors. Specifically, this study reports the revised scale's reliability, criterion validity, and factor validity with a broader sample. Unlike the previous study, which included only social workers, this one also includes psychologists and LMFTs. Methods: To assess the scale's internal consistency reliability and criterion and factorial validity social workers, psychologists and LMFTs were surveyed in Texas. All NASW members in Texas were invited to complete this questionnaire online by email, while 500 psychologists and LMFTs were randomly sampled from state licensing lists and invited to participate by postal mail. Confirmatory factor analysis procedures in AMOS 18.0 were used to assess the scale's factorial validity. Results: The sample included 946 social workers, psychologists and LMFTs, and there was no difference in overall scale score between each discipline (F[2, 943]= 2.67, p=.07). The overall scale had excellent internal consistency (α=.94), as did the self-efficacy (α=.93), attitudes (α=.91), and behaviors (α=.91) subscales. The feasibility scale also had fairly good internal consistency considering that it was only 5 items (α=.75). Criterion validity was established by demonstrating a relationship between the scale and reported prior exposure to EBP through continuing education and university courses (with coefficients ranging between .22 and .57). The confirmatory factor analysis goodness of fit indices suggested that the hypothesized four-factor structure of the scale had overall acceptable to good fit after the correlation of five theoretically justified error terms significantly improved the model's fit (X²/df=4.42, CFI=.90, RMSEA=.06, SRMR=.06). Conclusions and Implications: This study provides support for the reliability and validity of the shorter, revised EBP Process Assessment Scale with a more diverse sample of helping professionals, including social workers, psychologists and LMFTs. This briefer assessment instrument can be used to evaluate the outcome of training in the EBP model or in surveys to assess the views of practitioners toward EBP.