* noted as presenting author
John D. Fluke, PhD, Vice President, American Humane Association, Englewood, CO, Fred H. Wulczyn, PhD, Research Fellow, University of Chicago, Chicago, IL, Barbara Needell, PhD, Research Specialist, University of California, Berkeley, Berkeley, CA, Mark E. Courtney, PhD, Professor, University of Washington, Seattle, WA, Donald Baumann, PhD, Research Associate, Child Protection Research Center, Englewood, CO and Erin Dalton, BS, Deputy Director, Allegheny County Department of Human Services, Pittsburgh, PA
Purpose: The passage of the federal Adoption and Safe Families Act in 1997 led to the development of a comprehensive regulatory structure designed to assess the performance of state child welfare agencies. Based on outcomes for children expressed as safety, permanency, and well-being the federal government has conducted two rounds of Child and Family Service Reviews (CFSR). Each CFSR is a two-stage process consisting of a Statewide Assessment and an onsite review of child and family service outcomes and program systems. For the Statewide Assessment, the Children's Bureau prepares the data profiles that contain aggregate data on the State's foster care and in-home service populations. After the Statewide Assessment, an onsite review of the State child welfare program is conducted by a Federal-State team. The onsite portion of the review includes: (1) case record reviews; (2) interviews with children and families; and (3) interviews with community stakeholders. Failure to meet performance standards results in the requirement that states develop Program Improvement Plans (PIP) and may result in financial penalties if improvements are not realized. Despite efforts to improve the range of measures, the CFSR process has yet to take into account the underlying research principles an outcome based framework requires. This paper describes the current deficiencies and the methodological approaches that would improve the translation of research into practice.
Method: The paper begins with a critique of the existing methodology for conducting the CFSR reviews. Using state administrative data, alternative measures are presented, based on key research principles--including an awareness of the importance of tracking first time or initial services, inclusion of non-permanent exits, and an emphasis on longitudinal service trajectories. In addition, data are presented that demonstrate the utility of risk adjustment—which is critically important if states are to be compared with one another. These new measures would replace the existing data profile and be used to guide on-site review resources. States that perform exceptionally well or poorly on each new measure could be examined to assess what is contributing to those outcomes.
Results: The paper describes how performance standards can be reformulated, and a method for implementing the standards is proposed. The results draw on examples from states that have implemented such for purposes of internal and financial accountability. In addition, a process for ongoing improvement of the measurement methodology is described, which relies on analysis designed to rigorously fortify the relationships between programmatic or policy changes in relation to outcomes.
Conclusions and Implications: When monitoring outcomes for children in the context of complex systems, scientifically grounded methods offer valuable guidance to all concerned. Performance standards should not only address the outcomes of interest; safety, permanency, and well being, they should also be consistent with the best science available and a current understanding of how these measures best reflect the actual performance of the system. The approach to improving the CFSR methodology described connects research to practice to assess child welfare performance.
![[ Visit Client Website ]](images/banner.gif)