By: Courtney M Smalley MD, Margaret Sande MD MS MSHPEd, Juliana Wilson DO MPH, Resa E Lewiss MD
Lessons learned:
- Simulation-based competency assessments allow the standardized evaluation of complex patient management skills that are inconsistently captured through direct observation in the clinical environment.
- Low-fidelity simulation is sufficient to assess higher-order clinical reasoning as long as the patient case design and faculty facilitation are rigorous.
- Embedding technical skills within high-acuity clinical scenarios elevates the trainee assessment from an isolated task performance to integrated decision-making.
Point-of-care ultrasound (POCUS) is a powerful diagnostic tool integral to emergency department (ED) patient care. Although POCUS is a core competency for accredited residency graduation, the structured assessment of advanced competency remains resource intensive and non-standardized. While image acquisition and interpretation are typically evaluated during early residency training, fewer standardized methods exist to assess a senior resident’s ability to integrate POCUS interpretations into high-acuity clinical decision-making. We describe a six-year longitudinal education initiative using a simulation case to assess POCUS competency in senior emergency medicine (EM) residents.
Contemporary EM training frameworks emphasize progression from technical proficiency to real-time clinical integration of ultrasound findings. The Council of Residency Directors Academy of Emergency Ultrasound (CORD-AEUS) milestone framework highlights the incorporation of ultrasound into complex patient care scenarios as an advanced competency [1]. Many EM training programs have developed longitudinal or advanced ultrasound curricula to support this progression [2]. However, direct observation of integration skills during live ED shifts is inconsistent and dependent on case availability. Simulation provides a reproducible environment in which both technical execution and clinical reasoning can be observed under standardized conditions [3,4].
At the institution where we developed this education innovation, residents complete a three-week POCUS rotation during postgraduate year one (PGY-1) and a one-week advanced POCUS rotation during postgraduate year three (PGY-3). The authors previously published this curricular innovation [2]. The advanced rotation emphasizes ultrasound integration in unstable patients. As a capstone assessment, PGY-3 residents participate in a structured simulation involving a hemodynamically unstable patient in the first trimester of pregnancy presenting with hypotension, dyspnea, and presyncope.
The scenario progresses through staged clinical deterioration and requires learners to perform cardiac, lower extremity, and trans-abdominal and trans-vaginal pelvic POCUS examinations. Residents must recognize findings consistent with ectopic pregnancy, deep venous thrombosis, and pulmonary embolism and incorporate these findings into diagnostic prioritization and management decisions.
To enhance feasibility and reproducibility, the case is conducted in a low-fidelity simulation environment without reliance on a dedicated simulation center or dynamic simulation equipment. A standardized patient model is utilized for POCUS examinations. Residents demonstrate probe selection, positioning, and acquisition of technically adequate views. Once adequate technique is confirmed, faculty provide pre-recorded pathologic clips corresponding to each examination. A low-cost static task trainer constructed from a shoebox and soft packing material allows demonstration of transvaginal probe technique and appropriate imaging planes. A faculty member functions as bedside nurse, advancing the case, adjusting vital signs, and supplying laboratory or imaging data upon request.
Residents are evaluated in real time using a standardized direct observation tool aligned with ultrasound milestone domains: (1) image acquisition technique, (2) interpretation accuracy, and (3) integration of findings into clinical management. Learners are required to verbalize POCUS findings and explain how those findings alter diagnostic reasoning and therapeutic interventions. Assessment emphasizes clinical synthesis under simulated physiologic instability rather than isolated image recognition.
Over a six-year period, 102 PGY-3 residents completed the scenario. The case was reproducibly implemented annually with a consistent structure and assessment framework. Qualitative feedback consistently reflected perceived educational value, with residents reporting improved confidence in managing high-risk scenarios requiring ultrasound integration. Learners highlighted the opportunity for real-time feedback and deliberate practice in a psychologically safe environment. The scenario required minimal equipment and no dedicated simulation personnel, supporting feasibility and sustainability.
Qualitative post-simulation feedback consistently described the case as a high-yield assessment of real-time ultrasound decision-making. Residents valued immediate faculty feedback and deliberate practice in a psychologically safe environment. Many noted limited exposure to structured simulation-based ultrasound assessment during residency and reported improved confidence managing unstable patients requiring multimodal POCUS.
Simulation-based assessment offers several advantages for advanced POCUS evaluation. It standardizes exposure to rare but high-risk presentations and permits direct observation of decision-making processes that are otherwise difficult to assess objectively during clinical shifts. Prior literature supports simulation as an effective modality for procedural education and skills evaluation [3–5]. Our experience extends these principles to milestone-based assessment of ultrasound integration in senior EM trainees.
Limitations include implementation at a single academic center within a four-year EM residency program, which may limit generalizability. However, as EM programs are uniformly expanding into a 4 year training, we expect that this POCUS competency assessment will be of interest to medical educators. Outcomes were primarily observational and descriptive; formal statistical comparisons of checklist scores over time were not performed. Additionally, performance in a simulated setting may not fully replicate behavior in live patient care. Future investigation should incorporate structured scoring analyses and explore correlation with clinical ultrasound documentation and patient-centered outcomes.
In this longitudinal experience, a low-fidelity simulation case provided a feasible and sustainable method to assess advanced POCUS integration in senior EM residents. Embedding multimodal ultrasound decision-making within a crashing patient scenario enabled direct observation of clinical synthesis aligned with advanced training milestones. Simulation-based assessment may offer a practical strategy for programs seeking standardized evaluation of higher-level POCUS competency.
References:
- Lewiss RE, Pearl M, Nomura JT, et al. CORD-AEUS consensus document for the emergency ultrasound milestone project. Acad Emerg Med. 2013;20(7):740–745.
- Smalley CM, Sande M, Wilson J, Lewiss RE. An education innovation to improve point-of-care ultrasound skills in senior emergency medicine residents: a mandatory advanced rotation. J Medic Educ Training. 2017;1:001.
- Lewiss RE, Hoffmann B, Beaulieu Y, Phelan MB. Point-of-care ultrasound education: the increasing role of simulation and multimedia resources. J Ultrasound Med. 2014;33(1):27–32.
- McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Med Educ. 2010;44(1):50–63.
- Kumar A, Kugler J, Jensen T. Evaluation of trainee competency with point-of-care ultrasonography: a conceptual framework and review of existing assessments. J Gen Intern Med. 2019;34(6):1025–1031.
Photo curtesy of IStock
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa. For more details on our site disclaimers, please see our ‘About’ page