Admin note: This blog is Part 4 in a series! Scroll to the end of this post to view a list of the other posts.
—————————-
By: Martin Pusic (@mpusic) and Kathy Boutis (@ImageSim)
Alright, get ready for another non-intuitive connection between CBME and Learning Curves. In our previous posts, we made clear that learning is non-linear with inflection points and an asymptote that keeps increasing out to infinity. We discussed why this is and how to use this understanding to better frame our instruction. Now we turn to a key feature of CBME, the idea of setting competency standards, and discuss how this relates to the non-linear learning curve. And here’s the non-intuitive part: it doesn’t.
Standard Setting in CBME
To understand why standard setting should be divorced from the learning curve, we first need to examine what we mean by a competency standard. This is nicely defined here: Understanding Competency-Based Medical Education – AM Rounds (academicmedicineblog.org)1 and can be summarized as “An observable ability of a health professional” that can be measured to determine a performance level. This is the y-axis of a CBME learning curve (See Blog Post #1 in this series). Which performance level constitutes “good enough” is a socially constructed determination that involves multiple stakeholders, the most important of whom is the patient. The key contention is that the patient could care less where you are on your learning curve and how non-linear that path is. Are you competent? Have you met the performance standards? …are the only questions that matter. Whereas the learning curve is organically derived from the learning system — with all its inputs, affordances, engineering, and reinforcements – the competency standard is simply a point on the y-axis. An important one.
In Figure 1, we have drawn in three different putative competency standards (Lines A, B, and C). Each have tradeoffs in terms of the underlying learning curve. Line A, intersecting the learning curve before the second inflection point, leaves considerable efficient learning on the table (Pusic 20152). Even if the rate is slowing, the learner is still in a high yield learning phase where a relatively short increase in training effort would be rewarded with a considerable performance boost, on average. Contrast this with the competency standard at Line C. Here the implicit message is that the competency standard is the best practice possible – the community has judged that the standard needs to be high enough to be within the acceptable practice variation of the very best performers. This standard will be laborious to achieve.
Figure 1 – Standard Setting and the Learning Curve
Confusing Inflection with Mastery
We affectionately/facetiously refer to the Line B standard as the Senior Resident “False Dawn” Standard (having experienced it ourselves, if long ago). It is tempting to align the competency standard with the second inflection point of the learning curve. There are two important problems with such an approach. First, as mentioned above, the competency standard is a social construction that is in service of our patients and should ideally be set independent of the learning effort required. McGaghie and colleagues have rightly celebrated a mastery approach wherein the standard is set and the learning is designed in consequence (McGaghie 20153). The tail doesn’t wag the dog.
The second problem is more subtle but comes into play when the learner can influence when they call off their learning. At the second inflection point, the learner feels their learning slowing significantly. For example, by then a pediatric resident may have seen their 30th patient with an asthma exacerbation and feel they have mastered this presentation. But they are now in the phase of learning the rare, difficult things. They have yet to see (or misdiagnose) any of the “long tail” diagnoses that mimic asthma. They are now looking to learning things that occur 1/30 cases and not 30/30 cases. This next set of 30 cases will ideally lead to asymptotic learning, a type of learning that feels qualitatively different from the pre-inflection rapid phase of the learning curve. If this second inflection point is instead conflated with mastery, the risk is that the learner will adopt a routinized approach, one where they are no longer open to learning opportunities (Pusic et al 20194).
Milestones and the Learning Curve
We’ll close by considering how Milestones, Competency Standards and the Learning Curve intersect conceptually. As we’ve seen, in a Competency-Based system, we start with the end in mind, the competency standards we wish to achieve. These are socially constructed and usually reflect an agreed upon minimum performance on some meaningful metric. The competency standard ignores how hard something is to learn – it doesn’t care how you got there (Figure 1).
Milestones lay down developmental signposts on the non-linear learning path/curve. They imply progress on the way to a competency. You’ve made it part way, here’s where you’re going next in a (hopefully) predictable way. For example, a resident transitions from being required to have constant in-person supervision to being allowed to practice with only remote supervision. They are still far from the full competency standard, but a significant entrustment event has happened (Figure 2).
Figure 2 – Milestones and the Learning Curve
Summary
In this fourth blog post on Learning Curves in CBME we have made the following points. First, determining a competency standard is a process that is NOT based on the underlying learning curve. However, the learning curve can help determine how hard it will be for the learner to achieve various standards. Second, we warn again against considering the learning curve inflection point as being somehow related to a competency standard – if it is, it’s a coincidence. Finally, we have described how milestones can be considered as being significant educational process transitions that occur along the learning path.
Previous Blog Posts in this Series:
Part 1: Overture – Click here to read
Part 2: The early phases of learning– Click here to read
Part 3: The non-linearity of learning – Click here to read
Future Blog Posts in this Series
Part 5: Inter-Individual variability of learning curves – Now available – Click here to read!
Part 6: Summing up – Now available – Click here to read!
About the authors:
Martin Pusic, MD PhD is Associate Professor of Pediatrics and Emergency Medicine at Harvard Medical School, Senior Associate Faculty at Boston Children’s Hospital and Scholar-In-Residence at the Brigham Education Institute.
Kathy Boutis, MD FRCPC MSc is Staff Emergency Physician, Senior Associate Scientist, Research Institute at The Hospital for Sick Children and Professor of Pediatrics at the University of Toronto.
References
1.Englander R, T Cameron, A Addams, J Jacobs. Understanding Competency-Based Medical Education. AM Rounds 2015. Understanding Competency-Based Medical Education – AM Rounds (academicmedicineblog.org) Accessed Nov 18, 2022.
2. Pusic M, K Boutis, R Hatala, D Cook D. Learning Curves in Health Professions Education. Academic Medicine. 2015;90(8):1034-42.
3. McGaghie WC. Mastery Learning: It Is Time for Medical Education to Join the 21st Century. Academic Medicine. 2015;90(11):1438-41.
4. Pusic M, K Boutis, W Cutrer, SA Santen. How does Master Adaptive Learning ensure optimal pathways to clinical expertise? Chapter 16, pp 174-192 In: Cutrer W, Pusic MV eds. The Master Adaptive Learner, Amsterdam, NL. Elsevier Publishing Group. Nov 2019. Available at https://www.researchgate.net/profile/Martin-Pusic/publication/345393702_How_Does_Master_Adaptive_Learning_Ensure_Optimal_Pathways_to_Clinical_Expertise/links/5fa5bee5a6fdcc06241cbb58/How-Does-Master-Adaptive-Learning-Ensure-Optimal-Pathways-to-Clinical-Expertise.pdf
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page