Measuring Clinical Skill without Blowing Your Budget or Losing Your Mind: The Development of a Clinical Skill Assessment Tool Relevant to Military Populations

Men and women who return home from military service in Afghanistan and Iraq often need care for the “invisible wounds” (i.e., mental health conditions) that comprise the signature injuries of the current wars. Beyond DoD and VA resources, civilian behavioral health professionals are increasingly sought out to provide this care within the communities to which these men and women return. The need for a workforce equipped to meet the needs of the millions of Americans affected by military service is both apparent and urgent (Flynn & Hassan, 2010; Nissly & Turner, 2011). A large school of social work was awarded a grant to develop and test a curriculum to rapidly increase the number of behavioral health providers who are prepared to competently treat servicemembers, veterans, and their families. In addition to the content relevant to military culture, common injuries, and impact on family systems that are associated with military service, the curriculum aims to equip therapists with the clinical skills appropriate for practice with military-impacted populations. Measurement issues associated with assessing clinical skill among social work students include ceiling effects, field instructor reluctance to serve as a gatekeeper for the profession, and students’ clinical exposure being limited to the cases that present in their field placements (Badger & McNeil, 1998; Bogo, et al., 2007; Regehr, et al., 2011). The authors were challenged to develop a tool to assess clinical practice skill that could be delivered to MSW students on campus and enrolled virtually, as well as to advanced providers in communities across the country. The Military Clinical Skill Assessment (MCSCA) was developed as an alternative to the use of standardized patients to assess military-relevant clinical skills in a time and cost-efficient way. The authors will present the development of the MSCA with the intention to encourage discussion about this and other interactive assessment and teaching tools. The methods presented can be adapted to assess clinical skill development relevant to any population. The MCSA is an Objective Structured Video Examination (OSVE) that asks clinicians to respond to 14 video clips of a veteran/therapist interaction.  Literature on clinical skill assessment, combat veterans, and military-experienced clinicians were consulted to refine the script and accompanying prompts for face validity. A professional actor who is also a combat veteran played the role of the client in the 14 scenarios. Six therapist testers with varying levels of experience in clinical practice with military populations responded to the pilot MCSA. Military-experienced therapists coded the responses on the basis of appropriateness as outlined by best practices for advanced practice behaviors for clinical work with veterans, servicemembers, and their families. The resulting rubric asks independent raters to make judgments about practice behaviors in the domains of rapport building, appropriate communication and cultural awareness, unique issues relevant to military context, legal and ethical issues relevant to military clients, and culturally-informed client assessment. The MCSA’s utility in assessment of both new and experienced clinicians’ suitability for practice with military-populations, as well as a teaching tool for the classroom, will be discussed.