A Critical Review of Current Applications of HLM to Social Work Research

Background and Purpose: Hierarchical linear modeling (HLM) is an effective analytical tool to address the intraclass correlation issue of nested and clustered data. As a robust method, it has been increasingly employed in social work research. However the common pitfalls in using HLM and the problems found in articles using HLM in other disciplines raise our concern about research rigor in the applications of HLM. Using criteria outlined by Guo (2005), this review is conducted to find out the general status of using HLM in five major social work research journals from 2003 to 2007.

Methods: Five social work journals (i.e., Research on Social Work Practice, Social Work, Social Work Research, Social Service Review, and Journal of Social Service Research) were chosen because they are major journals in social work research with high citation rates among researchers. Articles published in these journals from 2003 to 2007 were examined. The review focused on eight major issues in applying HLM: assessment of intraclass correlation coefficient (ICC), testing assumptions about the normal distribution of the dependent variable, specification of random effects, measuring the variance explained (R-square), testing and interpretation of cross-level interaction, grand-mean centering or other centering strategies, presentation of results, and statistical power of the study.

Results: The review identified a total of 18 studies published in chosen journals that employed HLM. Common methodological problems of these studies are summarized below: (1) Of the 18 studies, only two reported ICC; (2) One study failed to use a multilevel Poisson model to address the nonnormal distribution of the dependent variable (i.e., event count or the number of recidivism); (3) Two studies mentioned the specification of random effects, but failed to provide details about the model fitting process; (4) Six studies reported R-square, but some did not report it accurately; (5) Only one study mentioned the cross-level interaction, but erroneously interpreted its meaning; (6) Two studies used grand-mean centered variables, and the remaining studies did not take advantage of centering transformation; (7) Statistical power analysis was not conducted for most studies, and some studies ran the risk of being underpowered due to using a small number of high-level units; and (8) Some studies did not present findings in an effective and parsimonious fashion (e.g., two studies did not provide a table reporting the estimated HLM results at all).

Conclusions and Implications: The increasing employment of HLM in social work research is encouraging. Researchers should be aware of statistical assumptions embedded in HLM, regularly diagnose their violations, take remedial measures if necessary, and report and interpret study findings accurately, efficiently, and effectively.