The Society for Social Work and Research

2014 Annual Conference

January 15-19, 2014 I Grand Hyatt San Antonio I San Antonio, TX

Competency Development for Leaders in Child Welfare

Sunday, January 19, 2014: 8:45 AM
HBG Convention Center, Room 008B River Level (San Antonio, TX)
* noted as presenting author
Robin Leake, PhD, Research Manager, University of Denver, Denver, CO
Shauna Rienks, PhD, Research Analyst, University of Denver, Denver, CO
Anna deGuzman, MS, Research Assistant, University of Denver, Denver, CO
Purpose: The Children’s Bureau’s National Child Welfare Workforce Institute (NCWWI) Leadership Academy for Middle Managers (LAMM) provides leadership competency-based training through pre-work, 6 days of residential training, and post-training supports. This study describes the development and testing of a leadership competency self-assessment tool that measures learning and transfer of learning as it evolves from knowledge-based to skill mastery through application. A common problem of self-assessments is inadequate testing for validity and reliability (Hinkin, 1995). To effectively use competency self-assessments as a measure of learning and transfer of knowledge, they must be developed in alignment with a strong competency-based training and administered to a large enough sample to allow for standard content or construct validation procedures. The current study aimed to develop a valid and reliable competency assessment for the NCWWI LAMM that measured learning and continued skill development in the year following training.

Method: LAMM curriculum designers and evaluators collaborated to develop a 28-item self-assessment (5-point Likert-style scale) that reflects the knowledge and skill competencies addressed in the LAMM training. The assessment was administered to LAMM participants to capture transfer of learning from a knowledge and understanding level of competency at pre-post (N = 223) to the gradual adoption of behavior and performance level of competency at 3 (N = 180), 6 (N = 151), and 12 months (N= 133) post-training.

Results: Reliability analyses on the self-assessment revealed strong internal consistencies at each time point: Cronbach’s α = .91 (pre-training), .95 (post-training), .94 (three months), .94 (six months), and .95 (12 months). Exploratory Factor Analyses indicated two underlying factors; here we focus on the one measuring general leadership skills (Cronbach’s α = .88, .90, .89, .86, and .89, at each timepoint). Using repeated measures ANOVA to test for change over time, we found significant competency gains from pre- to post-training, F(1, 390) = 1480.86, p < .001, η2 = .79; as well as significant gains from pre-training to the 12-month follow-up, F(4, 116) = 195.92, p < .001, η2 = .62. Over time, average leadership skills spiked at post-training then decreased slightly at each subsequent measurement point.

Conclusions and Implications: Findings from this study confirm the utility of a self-assessment tool for measuring change in leadership skills and indicate positive growth in competency among LAMM participants. Gradual “skill decay” is a common problem with training programs (Arthur, Bennett, Stanush, & McNelly, 1998) and speaks to the need for continued implementation of transfer of learning supports for participants. Based on the learning theory that participants begin understanding key constructs during a training event but acquire skills through practice on the job in a supportive transfer climate (Holton, Bates, & Ruana, 2000), we developed the self-assessment to focus on the knowledge and understanding level at pre- and post-assessments, and on performance level at follow-up assessments.  Discussion will include challenges associated with analyzing repeated measures designs with a measure that changes at different time points.