Admin note: This blog is Part 5 in a series! Scroll to the end of this post to view a list of the other posts.
—————————-
By: Martin Pusic (@mpusic) and Kathy Boutis (@ImageSim)
Up to now in this blog series we have described how the idealized learning curve works. We noted that learning really begins starts with a latent phase where the conditions of learning are established. After that learning takes off, though it does so in a non-linear fashion with decidedly unequal rates of learning, depending on where in the learning curve we find ourselves. And in the final phase, expert learning is shown to be a slowing, asymptotic phase where the going is not easy.
But does everyone experience the learning curve in the same way? What if, for example, 18 residents read the same 234 Ankle x-rays looking for fractures and learning with feedback after every case? (Pusic 2015) Would they each have the same learning curve and learn from feedback after every case? What do you think? Have a peek at Figure 1 where you’ll see…
Figure 1 – Inter-individual Variability in Learning Curves
… that the answer is an emphatic no!
In Figure 1 we have plotted out a form of learning curve for each individual resident completing the cases with feedback, and also plotted out the group average learning curve. For fun, trace the learning curves of a couple of the residents – what AMAZING variability in the lived experience of each individual! All doing the same radiographs.
This “spaghetti” plot has some interesting information. It shows that everyone starts off with vastly different prior knowledge (the y-intercept). And as they practice we see that the inter-individual variability — the spread between the worst and best performer — narrows considerably. This reflects a core educational truth – as a group learns not only does the mean performance rise but the reliability of that performance across the group also does.
What does this mean for Learning Curve theory?
Does this inter-individual invalidate learning curve theory? Not at all, it just means that to some greater or lesser extent, the learning progression holds at the group AND individual level. Notice that the group learning curve follows theory quite nicely. These learners are already primed to learn and so do not exhibit the latent phase and first inflection point we saw in earlier more ogive-shaped learning curves. Instead, they learn rapidly right out of the gate but in the decidedly non-linear fashion we’ve come to expect, with a gradual inflection that portends asymptotic learning. In the same way that the average of a list of numbers describes the “central tendency” of the numbers – an overall property of all the numbers — the group-level learning curve describes overall properties of the learning system even if no individual directly manifests that particular curve.
What we do know from multi-level modeling is that the group-level learning curve exerts/expresses a kind of gravitational pull on each individual, the strength of which we can measure empirically. These “multi-level” models have fixed effects (the group level curve) and random effects (the variability of individuals about the group level curve). For the gory math, you can refer to the references, but suffice it to say that we can usefully model both individual- and group-level learning phenomena using this framework (Reinstein 20212).
Let’s go one step further and make one of those sweeping education theory generalizations that are fun to debate but would make a KeyLime Methodology guru blush. Do you notice in Figure 1 the features of constructivism at play? Every individual starts with a unique level of prior knowledge. They take very different paths constructing their own radiologic mental framework. Each person seems to make sudden performance gains in different places. All of this crossed with a poker-like randomness in terms what each learner takes away from each radiograph case that has its own uniqueness. Fascinating (Yoon 20203).
What does this mean for CBME?
If there’s this much variability between people’s learning curves, what does that mean for CBME? Well, we think that’s actually the justification for CBME. The interindividual variability in how long it takes to learn something is baked right into the CBME educational philosophy.
Figure 2: Learning Curves from two learners under a CBME versus Time-Based learning framework
Consider Figure 2 where we show two learning curves from the same ankle radiograph study. CBME proposes a greater focus on outcomes rather than educational process. In a time-based standardized approach where learners are given a standardized experience (e.g. “complete 150 radiograph cases”) the tremendous inter-individual variability that we have seen is not taken into account in the educational design, leaving a gap between two individuals who have different starting points and different learning rates. In a CBME framework (e.g. “attain a minimum Sensitivity of 0.70”) the desired outcome is prioritized. In Figure 2, the learning curves show how the inter-individual variability is taken into account in setting the CBME instructional process (in this case having the person with the blue learning curve practice more) but is not in the time-based one.
Summary
While the learning curve theoretical framework models learning as a predictable, if non linear, learning progression, this reliably holds true in the aggregate and only to a varying degree at the individual level. Like mileage, your learning curve will vary from that of others. While this might be less tidy than we would like, it reflects certain core education truths: we vary in where we start any learning task; we vary in our learning rates and when we have our ‘aha’ moments. But we each eventually make our way to competency in a way that produces a reliable group. Doesn’t that sound like learning in residency?
Previous Blog Posts in this Series:
Part 1: Overture – Click here to read
Part 2: The early phases of learning – Click here to read
Part 3: The nonlinearity of learning – Click here to read
Part 4: Standard setting and the learning curve – Click here to read
The final Blog Post in this series (“Summing up“) is now available – Click here to read!
About the authors:
Martin Pusic, MD PhD is Associate Professor of Pediatrics and Emergency Medicine at Harvard Medical School, Senior Associate Faculty at Boston Children’s Hospital and Scholar-In-Residence at the Brigham Education Institute.
Kathy Boutis, MD FRCPC MSc is Staff Emergency Physician, Senior Associate Scientist, Research Institute at The Hospital for Sick Children and Professor of Pediatrics at the University of Toronto.
References
1. Pusic MV, K Boutis, R Hatala, DA Cook. Learning curves in health professions education. Academic Medicine. 2015;90(8):1034-42.
2. Reinstein I, J Hill, DA Cook, M Lineberry, MV Pusic. Multi-level longitudinal learning curve regression models integrated with item difficulty metrics for deliberate practice of visual diagnosis: groundwork for adaptive learning. Advances in Health Sciences Education. 2021;26(3):881-912.
3. Yoon JS, K Boutis, MR Pecaric, NR Fefferman, KA Ericsson, MV Pusic. A think-aloud study to inform the design of radiograph interpretation practice. Advances in Health Sciences Education. 2020;25(4):877-903.
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page