Reimagining the OSCE for Learning in Competency-based Education

By: Candace Pau, MD and H. Carrie Chen, MD, PhD

As health professions education (HPE) increasingly shifts toward competency-based frameworks, a debate has arisen about whether Objective Structured Clinical Examinations (OSCEs), a type of simulation-based assessment, have continued relevance.^1,2 Underlying this debate is the historical use of OSCEs for summative purposes and the evolving conception of assessment in competency-based education (CBE).³ In CBE, assessment heavily focuses on Assessment for Learning and use of multiple longitudinal data points to inform learner development. Frequent workplace-based assessments (WBAs) are emphasized for observation of authentic behavior, with a movement away from high-stakes summative examinations.^4,5 This paradigm shift has prompted many to question the role of traditional OSCEs in a competency-based program of assessment.³ However, we would argue that, in CBE, we need OSCEs more than ever.

Assessment for Learning

“Assessment for Learning” or “formative assessment” involves an iterative process that generates and uses feedback to effect further learning.⁶ To drive learning, assessments must provide rich and meaningful information about a learner progress along a developmental trajectory.⁷ OSCEs can contribute critically to a program of assessment by providing data about learner progress in ways and for contexts that other assessments cannot.⁸ In addition to allowing for systematic assessment of behavioral and higher order skills, it facilitates purposeful introduction of complexity (e.g., interpersonal dynamics, competing demands, logistical challenges) often difficult to control in WBAs. OSCEs may also enable richer feedback than workplace settings, where faculty observation and feedback are frequently constrained by time, patient factors, or the unpredictability of the clinical scenario.

One might argue that less structured simulation sessions designed only for learning could provide similar information about learner performance. However, the standardization and rigor with which OSCEs are designed and implemented can generate more specific and evidence-based feedback. OSCEs, by definition, should contain an adequate number of stations to support the generalizability inference in Kane’s validity framework; this generalizability allows the provision of meaningful performance data that can identify trends across domains and over time.¹⁰ Finally, lower-stakes (vs no-stakes) OSCEs framed as Assessment for Learning may allow for a modest amount of learner stress with a beneficial influence on learning by motivating learners to prepare for the exam.^11,12

Assessment of Newer Competencies

OSCEs have been widely utilized for assessment of competencies that center around communication and management of urgent/emergent clinical scenarios. However, OSCEs could also provide an ideal setting in which to observe how learners navigate newer emerging competencies, such as engaging with artificial intelligence to inform patient care or facilitating inclusive environments for patients and families. These and other more recently proposed competencies can often be challenging to assess in the authentic clinical environment due to absence of standardization, inability to guarantee learner exposure, patient safety concerns, and lack of faculty rater expertise or training.⁹ Moreover, the developmental trajectories for many of these nascent competencies are not yet well-delineated; OSCEs may therefore be a source of robust and valid data to inform the design and appropriate sequencing of instruction.

Admittedly, OSCEs, with their logistical complexity and rigorous standardization, are resource intensive. The unique benefits of OSCEs highlight the need to think strategically about where their data offers the most value relative to other assessment modalities. For instance, traditional OSCE-assessed skills of history-taking, physical examination, differential diagnosis, and documentation are generally well-assessed in the authentic clinical environment. OSCEs could instead be prioritized for clinical scenarios with added complexity or combinations of elements that would be difficult to consistently encounter in the workplace setting.

Summary

Traditionally, HPE has had a narrow view of OSCEs. The limited literature on formative OSCEs have focused on preparing learners for summative OSCEs.^13,14 The field is ripe for further investigation to better understand how learners perceive and engage with lower-stakes OSCEs and how they derive learning from it, and how learning might vary from WBAs or depend on the competencies assessed. So, rather than debating whether or not we should continue to employ OSCEs in HPE, our learners may be better served by turning our collective attention to answering a different question: for what purposes and in what contexts can we optimally leverage OSCEs to serve the ultimate goals of CBE?Medicine, that meant moving past the simple revision of syllabi to build a shared vision through structured engagement, active trust-building, and a completely faculty-driven competency-based redesign.

Refrences:

About the Authors:

Candace Pau, MD is a family physician, Associate Professor of Clinical Science, and founding Faculty Director of Simulation at the Kaiser Permanente Bernard J. Tyson School of Medicine. She is a doctoral learner in the UCSF-UMC Utrecht Doctoral Program in Health Professions Education.

H. Carrie Chen, MD, PhD is Associate Dean for Medical Education Research and Scholarship and Professor of Health Systems Science at the Kaiser Permanente Bernard J. Tyson School of Medicine.

The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa. For more details on our site disclaimers, please see our ‘About’ page