Development and Evaluation of Advance Care Planning Implementation Assessment Tool

Background and Purpose:
Approximately 1.2 million people live in over 15,000 certified nursing homes in the U.S., where nearly 30% of older adults spend their final days. The high volume of deaths in these settings has made improving end-of-life care a national priority. Advance care planning (ACP) is a key strategy for ensuring that care aligns with residents' wishes, particularly when they can no longer speak for themselves. However, the lack of a universally recognized standard for ACP implementation contributes to inconsistent practices and suboptimal outcomes. This study aims to develop and evaluate a tool to assess the quality of ACP implementation in nursing homes.

Methods:
Using a three-round Delphi consensus process, 18 clinical experts from social work, nursing, and medicine refined an initial tool developed from a systematic review. Round one included five focus groups, followed by two rounds of electronic surveys with endorsement ratings (1 = endorse, 2 = endorse with modification, 3 = not endorse). For psychometric analysis of reliability and validity, 31 interdisciplinary clinicians rated three vignettes representing high, medium, and low ACP implementation quality. The primary measure was a 19-item binary tool (Yes = 1, No = 0, Not Sure = missing) covering two dimensions: nursing home structural support and standardized implementation procedures. A global item rated overall ACP quality on a 5-point scale served as a proxy measure.

Results:
Focus group content analysis yielded six categories for tool refinement, including clarifying the scope of ACP, process components, team collaboration, training needs, documentation details, and alignment with federal/state regulations. Based on the identified suggestions from Delphi panelists’ suggestions, the revised tool consists of 19 binary items (1=Yes, 0=No, and Not sure = .) under two major dimensions: structural support and standardized implementation procedures. Consensus was reached after two survey rounds (Interquartile range = 0, median = 1, agreement >80%). The tool demonstrated excellent internal consistency (α = .93 for structural support, α = .95 for implementation procedures) and high inter-rater reliability (ICC = .80). Validity was supported by strong convergent validity (r = .81, p < .001) and known-group validity (χ²(2) = 62.7, p < .001).

Conclusions and Implications:
This was the first tool to assess ACP implementation quality at the nursing home level, incorporating both structural and procedural components. It addressed a critical gap in the field by offering a standardized approach to evaluating ACP practices, with implications for practice, research, and policy. For researchers, the tool can provide a means to quantitatively measure ACP implementation quality, which can be used to compare implementation outcomes across programs, settings, and populations, contributing to a more robust evidence base in the implementation science of ACP. From a policy perspective, the tool can aid in evaluating the impact of regulatory and funding initiatives aimed at promoting ACP. For practice, by identifying strengths and gaps in both structural support and procedural implementation, facilities can target specific areas for quality improvement, staff training, and policy alignment. This can lead to more consistent, resident-centered end-of-life care that honors individuals' preferences.