The Pressure of Selection: Reconciling CBE and Grades

By: Holly Caretta-Weyer, MD, MHPE

If you are or have spent time with a residency or postgraduate program director in recent years, you have likely heard the following or something similar: “There are so many applicants, and I can’t even begin to tell them all apart!” As competency-based education (CBE) has permeated the halls of our medical training institutions, many have begun to abandon the traditional assessment metrics used to compare students to one another in the postgraduate selection process, including standardized test scores and clerkship grades.

Residency program directors, left with very little by which to compare applicants, have bemoaned this change as making it impossible for them to differentiate between the hundreds, if not thousands, of applicants to their training programs.¹ As such, they have turned their attention to alternative measures such as research publications, leadership experiences, or other unique areas of expertise by which to compare students as they apply for postgraduate positions. This has created a shadow economy of effort, pressuring students to spend more time in alternative pursuits, taking their focus away from becoming excellent and well-prepared physicians able to care for patients on day one of their postgraduate training.²

As such, program directors have turned to medical schools and begged for a return to grades. However, knowing that norm-referenced grades place undue pressure on students to hide their areas for growth and hinder the adoption of CBE in its intended form, many schools have pushed back on this.³ Alternatively, many others, citing concerns about their students obtaining postgraduate residency positions and not wanting to harm those prospects, have begun to move the needle back toward having grades.

This presents an even greater challenge, given that we know norm-referenced grading is often biased and places students who are underrepresented in medicine at a disadvantage. These biases often stem from socioeconomic barriers to accessing resources, those who have been exposed to medicine adjusting to the clinical environment more quickly, and traditional views of professionalism perpetuating stereotypes of who simply seems more like a physician.^4,5

Where then does this leave us? The pressure of selection inherently requires the comparison of students, while CBE demands students compare themselves to a standard, resulting in significant similarities between them upon graduation, as intended to meet the needs of patients.

When considering bias in assessment, the recent Macy conference recommendations state that optimal assessment is competency-based and equitable.⁶ In our mandate to return to grades, our team considered this charge. In order to achieve a system in which both competency-based assessment and grades could coexist, we would need to rethink the traditional clerkship grading system. This would require us to engage interest holders from both medical school and postgraduate residency training, as well as students and patients. By prioritizing preparedness for day one of residency and fairness in assessment, could we find a way to incentivize the right things, track them longitudinally, and reward students in domains that were more relevant both to program directors and to patients?

This has led us to create a system whereby we will be awarding grades not at the individual clerkship level (beyond the current pass/fail) but longitudinally in relevant domains across clerkships. These domains reflect areas that patients and postgraduate program directors alike prioritize in what they are looking for in someone who is prepared for the patient care demands expected of our learners on day one of residency training. These include domains like adaptive expertise, inter-professional teaming, systems-based advocacy, and clinical reasoning. By getting away from abstract concepts such as “patient care” and “professionalism,” we are able to be more granular about where students are excelling. And, we can track this over time across a number of different contexts and clerkships. We intend to reward growth and development over time, including seeking and responding to feedback.

These “grades” will be criterion-referenced. Students will be able to attain any number of areas of distinction if they meet the criteria and demonstrate growth across their clerkships. In true CBE fashion, we will employ a competency committee to review the programmatic assessment data from direct observation, multi-source feedback, electronic health record data, and other sources to make informed decisions around progression and domains of distinction. And to ensure fairness, we will audit all of these processes for bias and make adjustments where needed. By doing all of these things, we hope to be able to implement a truly competency-based system while also providing meaningful information for postgraduate selection and ensuring preparedness for day one patient care in residency training. Stay tuned for updates to see how we get on as the process evolves!

Refrences:

About the Author:

Holly Caretta-Weyer, MD, MHPE is Associate Professor of Emergency Medicine and Associate Dean for Admissions and Assessment at the Stanford University School of Medicine. Her work focuses on leveraging precision education principles in selection and assessment across the continuum of medical education with an eye to patient and learner outcomes as well as the implementation of competency-based education within emergency medicine training.

The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa . For more details on our site disclaimers, please see our ‘About’ page

Refrences:

About the Author:

Share this:

Like this:

Discover more from ICE Blog