Competency Development for Leaders in Child Welfare
Method: LAMM curriculum designers and evaluators collaborated to develop a 28-item self-assessment (5-point Likert-style scale) that reflects the knowledge and skill competencies addressed in the LAMM training. The assessment was administered to LAMM participants to capture transfer of learning from a knowledge and understanding level of competency at pre-post (N = 223) to the gradual adoption of behavior and performance level of competency at 3 (N = 180), 6 (N = 151), and 12 months (N= 133) post-training.
Results: Reliability analyses on the self-assessment revealed strong internal consistencies at each time point: Cronbach’s α = .91 (pre-training), .95 (post-training), .94 (three months), .94 (six months), and .95 (12 months). Exploratory Factor Analyses indicated two underlying factors; here we focus on the one measuring general leadership skills (Cronbach’s α = .88, .90, .89, .86, and .89, at each timepoint). Using repeated measures ANOVA to test for change over time, we found significant competency gains from pre- to post-training, F(1, 390) = 1480.86, p < .001, η2 = .79; as well as significant gains from pre-training to the 12-month follow-up, F(4, 116) = 195.92, p < .001, η2 = .62. Over time, average leadership skills spiked at post-training then decreased slightly at each subsequent measurement point.
Conclusions and Implications: Findings from this study confirm the utility of a self-assessment tool for measuring change in leadership skills and indicate positive growth in competency among LAMM participants. Gradual “skill decay” is a common problem with training programs (Arthur, Bennett, Stanush, & McNelly, 1998) and speaks to the need for continued implementation of transfer of learning supports for participants. Based on the learning theory that participants begin understanding key constructs during a training event but acquire skills through practice on the job in a supportive transfer climate (Holton, Bates, & Ruana, 2000), we developed the self-assessment to focus on the knowledge and understanding level at pre- and post-assessments, and on performance level at follow-up assessments. Discussion will include challenges associated with analyzing repeated measures designs with a measure that changes at different time points.