Development and Initial Validation of a Human Services Inventory (HSI) for Child Protective Services (CPS) Workers

Background and Purpose: This paper describes the results of initial research to develop a new Human Services Inventory (HSI) for new CPS job applicants in Texas. Included are the results of multiple validation studies and initial reliability estimates that support the psychometric characteristics of the HSI and its use for employee selection.  

Problem Statement: There are no research findings to establish the initial psychometric characteristics (validity and reliability) of the new HSI.

 

Major Objective: The major objective of this study was to develop an initial, psychometrically sound HSI for all new CPS job applicants.

Research Questions:

 

1. What is the measurement structure and construct validity of the new HSI ?

2. Do initial research findings support the content and criterion-related validity of the HSI?

3. Do initial research findings with the HSI support its measurement reliability?

Methods: The study utilized mixed methods (quantitative and qualitative) to examine the initial psychometric characteristics of the new HSI. Items for the HSI were statements of elements of CPS practices grounded in recent research studies of new employee selection and retention. A set of 56 professional practices related to employee retention were content classified by expert judgments. Subsequently, four focus groups of highly experienced CPS professionals (N=20) used a standardized format (Likert scale) to make judgments about item clarity, job relatedness, and potential for affecting performance outcomes with clients (children and families).   

A large, random sample of inexperienced CPS job applicants (N=538) and experienced CPS staff (N=624) was selected from these populations using the state agency data base. These participants completed a web-based survey of CPS practices judged as job-related and outcomes-based. (i.e., important for client outcomes).  A series of Principal Components Analyses (PCAs) was completed on the survey data base.  Cronbach Alpha reliability estimates were then computed for each HSI component. ANOVAs were computed to compare HIS component responses for inexperienced and experienced respondent groups.    

Key Results of the study were as follows:

 

• The focus group data provided strong support for the content validity of the HSI items.  

• The PCA results showed that only eight of 56 items loaded at or below .33 and a large number of item/component loadings were at or exceeded .50.

• Five HIS measurement components were identified through the PCAs (# of items and Alpha coefficients are in parentheses):

 

1. Receptivity to Client Needs and Professional Relationships (17, .94)

2. Professional Commitment (11, .87))

3. Work Stress Tolerance and Resilience (7, .88)

4. Client Interactions and Effects (10, .93)

5. Intent to Remain Employed (10, .79)

ANOVA comparisons between inexperienced and experienced groups using the HSI components showed higher ratings for the experienced group than for the inexperienced group (p<.000)

Conclusions and Implications:  The results of this study show that the new HSI to be used for employee selection in Texas demonstrates adequate, initial validity (structural, content and criterion-related) and reliability, to meet the purpose for which the HSI has been designed.