Using the Pediatric Symptom Checklist for Child Mental Health Screening. Can Type 2 Error Be Reduced Through a Multi Factor Scoring Methodology?

Purpose: This study investigates the psychometric properties of the Pediatric Symptom Checklist (PSC) and the impact of implementing an alternative four factor scoring methodology on Type 2 error.  The PSC is a broad based mental health screener designed to be implemented in a physician’s office, and has been adopted by New York State Child and Family Clinic Plus as a universal mental health screening instrument. For the purpose of screening, an instrument needs to identify individuals who are likely to be aided by further assessment, while minimizing those incorrectly identified.  Validating the ability of the PSC to identify children and adolescents who are at risk for emotional or behavioral issues would allow schools and community agencies to better develop/implement mental health screening and intervention programs.

Method: This study is a secondary data analysis using historical chart review for data collection purposes. A sample of 2760 clinic based cases were collected and cases randomly divided into three equal samples for data analysis using MPlus6. A four factor model was chosen using EFA and ESEM and a new scoring methodology was developed utilizing ROC curves to determine cutoff scores.  This scoring methodology was tested on the third sample and results compared to the single factor published scoring method, these results are presented. The determination of Type 2 error for this sample was done based upon the results of clinical assessment and compared to the initial PSC results, as the entire sample received a clinical assessment after completing the PSC.

Results: The 4 factor scoring methodology was chosen based on the goodness of fit and the clinical significance of the items in each factor. The statistics for the final four factor solution in ESEM resulted in: Chi-Square=787.767, DF=413, P-Value=0.0000, RMSEA=0.033, CFI=0.975 and TLI=0.966.  When scoring the entire sample using the published cutoff score, Type 2 error occurred in 35% (n=960) of the cases. Implementing the new four factor scoring methodology reduced the amount of cases falling into Type 2 error to 2% of the total cases (n=48).  The utility of the PSC as a screening instrument could be greatly improved by utilizing this subscale scoring method.

Conclusions and Implications: Utilizing the PSC as a universal screening instrument with a one factor scoring method is not adequate. Providing a mental health screening that is not adequate to correctly identify those in need of continued assessment does nothing except fail our children and adolescents who are most vulnerable and build barriers to care. Universal screening of children and adolescents for mental health issues is more than just a good idea, it is a necessity. Using valid and reliable screening instruments is required for this type of screening to be successful. Continual review and improvement of the psychometric properties of screening instruments is imperative. By continuing to study instrumentation and make changes based upon the current findings, improvements in identification and treatment can be made.