Ensuring Equity and Fairness in Assessment

By: Daniel J. Schumacher (@DrDanSchumacher)

The maturation and advances we have seen in medical education assessment over the past few decades are remarkable.  We are no longer tethered to high stakes single moment in time knowledge exams.  Indeed, these tests have found themselves under scrutiny in recent years.  This critique is likely well-placed.  We know that knowledge alone is not sufficient and that several other 21st century competencies, such as interpersonal communication skills, empathy, and trustworthiness, are central to being a healthcare professional.  So, it is illogical that knowledge exams alone would continue to be given the singular importance they have been for several decades.  Furthermore, these exams have been shown to have bias and unfairness built into them, curtailing the careers of those historically underrepresented in medicine.

While I offer criticism for our history of placing supreme value in high stakes single moment in time exams, I am not naïve.  Work-based observational assessment and valuing the subjective and collective in assessment that we have seen gain increasing centrality in our assessment efforts can also be done in ways that lack equity and fairness and allow unwanted, harmful biases to propagate and even run amok.  To avoid this, I offer four guiding principles:

Focus on formative over summative

While we need to make summative assessment decisions (such as progression to subsequent years of training, promotion to being a supervisory resident, and graduation), our primary assessment focus much of the time should be formative.  This shifts the stakes of assessment to be and feel lower for trainees, thereby setting the stage for learning environments that trainees feel safe and supported in.

Focus on development over judgment

In keeping with a focus on formative assessment, the bulk of daily assessment efforts should also help learners take the next steps in their development.  This seeks to help them do what they are not yet ready to do rather than simply levying a judgment that they are not yet ready.  This is good for their sense of safety, support, and care received and also centers patients by preparing learners to be able to better meet their needs rather than simply noting they are less ready to meet those needs.

Focus on criterion-referencing and eliminate norm-referencing

Criterion-refencing seeks to set a standard to achieve.  Everyone can meet that standard, no one can meet that standard, or anything between these extremes.  If the goal is readiness to provide a certain standard of care to patients, our goal should be for everyone to meet that bar.  If everyone is above the bar, comparing their performance to one another (norm-referencing) will not meaningfully change the care patients receive.  Rather, these comparisons will open the door to bias, favoritism, inequity, and bad feelings about oneself for those on the lower ranges of the normal distribution.  Furthermore, if everyone is below that bar, it is nonsensical to compare their performance to one another.  From the patient perspective, all of those individuals are not ready to provide the care patients need and deserve.

Focus on transparency and eliminate opacity

In medicine, inequity and unfairness lurks in the shadows of private meetings among leaders, hallway conversations with those who are disadvantaged not privy to inclusion, and the ways unspoken norms play out in systems.  To center equity and unfairness, these opacities must be eliminated.  They advantage the already advantaged and disadvantage the already disadvantaged.  All aspects of assessment, from how and why we collect the data we do to how and why we use that data the way we do, must be transparent to learners.  Furthermore, the design, implementation, and maintenance of our assessment practices must include trainees – nothing about them without them.

These four principles are central to competency-based medical education (CBME).  In my view, we simply have to get the implementation of CBME right because it centers high quality equitable education for learners that prepares them to provide high quality equitable care that patients need and deserve.

About the author: Daniel J. Schumacher, MD, PhD, MEd, is Director of the Education Research Unit at Cincinnati Children’s Hospital Medical Center and a tenured professor in the University of Cincinnati’s Department of Pediatrics

The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page