- To what extent do observers recognize the target behaviors during simulation?
- What impact does real-time feedback have on the perception of knowledge after simulation compared to a pre-assessment perception of knowledge?
- What impact does real-time feedback have on the perception of skill after simulation compared to a pre-assessment perception of skill?
- How effective do participants feel the simulation training format of real-time, quantitative feedback is?
- What impact does the in-person vs. virtual meeting platform have on recognition of the target behaviors during simulation?
Methods: Each trainee completed real-time digital rating forms indicating whether each skill was effectively demonstrated for every section of the simulation lab. Then feedback was provided in real-time using concurrent expert trainer ratings on the same forms and expert observations of simulation performance.
A pre/post format was used to measure changes in perceptions of knowledge and skill in trainees. These data were correlated with self-efficacy reports from each observer after the simulation to explore the degree to which trainee performance correlated with self-efficacy. Differences in trainee performance and perceptions of learning effectiveness between in-person and online delivery formats (pre- and post-COVID-19, respectively) were analyzed.
Results: Trainee perceptions (N=300) of knowledge and skill significantly increased from pre to post. Findings suggest this assessment of peer observers during simulation increases engagement in the simulation lab and provide a more quantitative understanding of whether observers are learning to recognize when best practice skills are demonstrated. Of particular interest, moving the simulation to an online meeting platform did not negatively impact these results. All indicators of knowledge, skill, confidence, and perceived effectiveness remained relatively level post-Covid-19 compared to in-person simulations before.
Conclusions and Implications: This simulation method of increasing observer engagement and learning assessment enables trainers and educators to provide meaningful feedback to simulation participants who are observing, whether in-person or virtual. Such specific and quantitative feedback enables the broader effect of simulation beyond the individual learner “on the stage.” Impact on the transfer of learning will be explored in future studies.