By: Mary Ott (@mary_mott2)
Residents may struggle to shoulder many assessment burdens in CBME programs as the result of additional workload and anxiety.1 Some of these burdens are not unique to residents, however. Chronic work overload causes physician burnout.2 System leaders must be watchful for sources of additional work in providing a competency-based education and mindful of ways to lighten the load. It’s time to cast a light on one lurking below the surface of your program’s culture that may be nine-tenths of the assessment burden iceberg. Failure to avoid it is a costly mistake.
Is Technology serving the process or shipwrecking it?
Teachers need assessment information to tailor instruction to individual needs. Learners need actionable feedback to inform their development and set mastery goals. Competence committees need rich sources of evidence to make entrustment decisions. To serve the collection and analysis of all this data, assessment forms in e-portfolios should be easy to request, complete, and review – not cumbersome, difficult, and time-consuming. Unfortunately, that’s been the experience of many faculty and residents in my context. While the reasons are complex, there are some common denominators.
First, while technology platforms vary institutionally, they are generally not compatible with mobile devices. Mobile-ready technology would go a long way in reducing the time burden of requesting and completing EPA assessments – and support the goal of providing contextually rich and actionable feedback for learning.3,4 Is your system budgeting for technology that includes mobile solutions? This should be an essential line item.
Then there’s the problem of how to make sense of the burgeoning amounts of assessment – a database on its own is a blackbox of information. Dashboards for residents and competence committees are essential tools for visualizing data and tracking learning progression.5 But whose dashboard is developed first? In some institutions, prioritizing entrustment decisions in the development of dashboards for competence committees has come at the cost of providing developmental information for learners.6,7
This discounts the value of assessment for learning, but the problem goes deeper. Deficits in learner-centred technology compound sources of anxiety for trainees. Lack of timely feedback and tools for understanding progression makes them less competent, autonomous, and supported as learners. They can feel vulnerable and de-motivated, under a panopticon of surveillance about ‘getting’ EPAs without meaning or purpose. System leaders must price in adequate resources to support technology for CBME. Failure to do so comes at the cost of resident learning and well-being.1
But is costly technology development the answer? Some programs are adapting to the barriers posed by unwieldy and poorly supported technologies with low-budget solutions available to any institution, such as online survey systems for assessment forms, spreadsheets for data archiving and review, and open-source software for dashboards.8 And has your program considered using voice to text on mobile devices to capture real-time assessment? Or video recording procedural elements to document multimodal feedback? Adapting technologies that are already user-friendly, accessible, and cross-functional to CBME purposes may well be the answer to lightening assessment burden for your faculty and residents.
Humanizing CBME
Culture becomes with technology, so look for human-centred designs. But technology is also no substitute for the human touch in education. Faculty advisors or coaches are promising innovations in CBME that support the Core Components Framework9 by helping residents to consolidate multi-source feedback and plan next steps. Longitudinal coaching can play a vital role in caring for learners, supporting their development over time, and fulfilling the promise of CBME to improve medical education. So, don’t forget to care for your educators too. This may include stipends or other performance incentives for coaches and competence committee members to free up time to make this essential but labour-intensive work sustainable. The cost of physician burnout is also on the line.10
About the author: Mary Ott, PhD, is Centre Researcher, Centre for Education Research & Innovation at the Schulich School of Medicine, Western University
References:
1. Ott MC, R Pack, S Cristancho, M Chin, JA Van Koughnett, M Ott. “The most crushing thing”: Understanding resident assessment burden in a competency-based curriculum. J Grad Med Educ. 2022;14(5):583-592.
2. Yates SW. Physician stress and burnout. Am J Med. 2020;133(2):160-164.
3. Young JQ, R Sugarman, J Schwartz, M McClure, PS O’Sullivan. A mobile app to capture EPA assessment data: Utilizing the consolidated framework for implementation research to identify enablers and barriers to engagement. Perspect Med Educ. 2020;9(4):210-219.
4. Chen JX, E Kozin, J Bohnen, B George, DG Deschler, K Emerick, ST Gray. Assessments of otolaryngology resident operative experiences using mobile technology: A pilot study. Otolaryngol Head Neck Surg. 2019;161(6):939–945.
5. Sonnenberg LK, P von Hauff, L Lemieux. Electronic portfolios for assessment in your postgraduate medical education program: Essential questions to ask when selecting a platform for competency-based medical education (CBME). MedEdPublish. 2017;6(2).
6. Ott MC, S Cristancho, T Apramian, & K Roth. Unintended consequences of technology in competency-based education: A qualitative study of lessons learned in an OtoHNS program. J Otolaryngol Head Neck Surg. (in review).
7. Thoma B, V Bandi, R Carey, D Mondal, R Woods, L Martin, et al. Developing a dashboard to meet competence committee needs: a design-based research project. Can Med Educ J. 2020;11(1):e16–34. doi: 10.36834/cmej.68903
8. Carey R, G Wilson, V Bandi, et al. Developing a dashboard to meet the needs of residents in a competency-based training program: A design-based research project. Can Med Ed J. 2020;11(6):e31-e45.
9. Van Melle E, JR Frank, ES Holmboe, et al. A core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94(7):1002-1009.
10. A profession under pressure: results from the CMA’s 2021 National Physician Health Survey. Canadian Medical Association. Accessed August 29, 2022. https://www.cma.ca/news/profession-under-pressure-results-cmas-2021-national-physician-health-survey, https://www.cma.ca/news/profession-under-pressure-results-cmas-2021-national-physician-health-survey
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page