How to Assess and Measure Transfer of Learning in Child Welfare: Results from a Scoping Review

Background and Purpose: The quality of child welfare services hinges on staff building competency in core skills including outreach and engagement, assessment, critical thinking, and decision-making. Yet we know that without additional reinforcement strategies, the likelihood of Transfer of Learning (ToL) from training programs alone is low. Importantly, understanding how well skills are implemented requires strategic methods for measuring how well and how often skills are used in the real world.

Methods: Using data captured through a scoping review of published research between 1998-2020, this paper identifies and compares different methods used for assessing ToL as reported in 21 child welfare related studies. Utilizing a descriptive analytic framework, publications were examined for type(s) and timing of assessment and critically evaluates the pros and cons of each method.

Results: Of the publications examined (n = 21), 7 distinct types of strategies were utilized to assess ToL. These strategies include the use of observational skills assessments (n = 10), audio and/or video recordings (n = 6), self-assessments (n = 6), composite scores (n = 4), surveys (n = 3), documentation reviews (n = 2), and focus groups (n = 1). Pre-ToL strategies include the use of a pretest questionnaire and an online introductory module. ToL was assessed between 1 months and 24 months, with most clustered between 2 – 6 months. Stronger assessments included the use of validated measures e.g., The Child Abuse Interview Interaction Coding System, the MITI coded by certified raters with participants interviewing standardized clients, and the Training Transfer Inventory). Several studies incorporated client feedback about their observation of the use of skills and/or self-reports of their achievement of family level outcomes because of implemented skills. In a few studies, qualitative assessments asked participants to provide specific examples about how skills were used and to report on the client response to the use of skills. Less rigorous measures included self-reports from participants about their confidence and self-perceived competence using skills with real clients.

Conclusions and Implications: Future research should consider implementing replications of similar assessment measures to strengthen the reliability and validity of ToL assessment measures. Ideally, multiple assessment over time would increase the likelihood that competent use of skills is implemented with some degree of consistency over time.