By: Andrew K. Hall MD, MMEd, FRCPC, DRSPSC
The implementation of Competence-By-Design (CBD) in specialist postgraduate medical education by the Royal College of Physicians and Surgeons of Canada has been ongoing for the last decade1. Integral to this implementation has been a plan for intentional concurrent program evaluation, built around three main goals: supporting successful implementation, building evidence of impact, and fostering a community of practice around CBD program evaluation2. To foster the third goal – an active community of practice – the Royal College recognized that a multi-pronged approach was required with consideration of local, regional and national elements. The aim was to catalyze program evaluation efforts both in collaboration with the Royal College, and by engaged teams from across Canada.
The cornerstone of this strategy has been the annual National CBME Program Evaluation Summits3, which have been a pre-conference activity of the International Conference on Residency Education (ICRE) since 2017. The first in-person summit featured the sharing of potential CBD focused program evaluation strategies and discussions aimed at tackling the question of how to effectively evaluate this broad system change initiative. Over time, the summit has transitioned to a virtual event with abstract presentations from those engaged in program evaluations works related to CBD, plenary presentations from educational leaders and institutions describing their program evaluation approaches, and the outcomes of program evaluation works driving CBD program and system adaptations.
Our most recent event, the 2025 CBME Program Evaluation Summit, convened faculty leaders, researchers, residents, and education scholars from across the country to exchange insights and strategies for advancing the evaluation of CBD. The event highlighted how institutions are maturing in their approaches to evaluation, shifting from compliance and fidelity-focused exercises to holistic, learner-oriented, and system-embedded processes that drive educational improvement and responsive adaptations. Multiple Canadian institutions shared their approaches and most recent findings. Highlights included the identification of institutional culture and relationships as critical elements of implementation success, a focus on competence committees as the cornerstone of CBD process and ideal target of improvement-focused effort and evaluation, the identification of persistent tensions between measurement and learning particularly from the learner perspective, and the further characterization of assessment pressures and negative impact on wellbeing through collaborative resident-focused evaluation.
The abstract session then highlighted an array of program evaluation projects reflecting an impressive breadth of scholarship and a variety of innovative program evaluation methods. (The full 2025 CBD PE Summit program can be found here) . Overall, while previous abstracts may have focused more on challenge identification and characterization, this year’s seemed to be consistently solutions-oriented. There were multiple important themes which could be identified from these presentations. The first key area was a focus on assessment system refinement. Several projects investigated how changes to assessment forms and processes influence feedback quality and faculty engagement, revealing that culture and development, not just form design, drive change. Next were studies focused on the experience of residents in the learning environment, emphasizing the importance of psychological safety, wellness, and trust within CBME ecosystems, underscoring that the learning environment profoundly shapes the success of competency-based programs. Other studies focused on data science, exploring models and processes to automate narrative EPA analysis, offering scalable and consistent evaluation mechanisms. Simulation integration was also highlighted as an underutilized potential for assessment, advocating for its broader use to enhance learning and standardization. Finally, several programs developed logic models to link inputs, activities, and outcomes, promoting transparency and continuous improvement in CBD programs. The closing plenary was then focused on program evaluation applied in practice, bringing all of these above threads together, demonstrating how program evaluation can inform responsive adaptations at the individual program, PGME, and specialty committee levels
Reflecting on the entire event, I can say that we are truly lucky in Canada to have such an engaged community of practice related to program evaluation of CBD. This work and this community are a major driver of the evolution of CBD in Canadian specialty postgraduate training. The 2025 CBD Program Evaluation Summit underscored that evaluation in CBME is entering a new phase of sophistication and reflection. Canadian programs and institutions are increasingly recognizing evaluation as a cornerstone of educational quality improvement and cultural transformation. Collectively, our community is engaging in effective program evaluation as a shared act of curiosity, collaboration, and commitment that is sustaining the ongoing evolution of specialty education in Canada.
For more information, please see: https://www.royalcollege.ca/en/standards-and-accreditation/competence-by-design.html
Acknowledgements:
We are greatly appreciative of the many members of the medical education community in Canada who attended the CBME Program Evaluation Summit this year and previously!
The CBD Program Evaluation Summit is jointly run by the CBD Program Evaluation Operations Team from the Royal College of Physicians and Surgeons of Canada including the following members:Andrew K. Hall, Damon Dagnone, Anna Oswald, David Rojas, Tim Dalseg, Tanya Horstley, Warren Cheung, Stanley Hamstra, Stacey Brzezina, Sinthiya Selvaratnam, Christa McMillan, Ajani Asokumar
Refrences:
- Frank JR, Karpinski J, Sherbino J, et al. Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education. Perspect Med Educ. 2024;13(1):201-223.
- Hall AK, Oswald A, Frank JR, et al. Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes. Perspect Med Educ. 2024;13(1):95-107.
- Competence By Design Program Evaluation. https://www.royalcollege.ca/en/standards-and-accreditation/competence-by-design/program-evaluation.html. Accessed November 12, 2025.
About the Author:
Andrew K. Hall MD, MMEd, FRCPC, DRSPSC, is the Vice-chair Education in the Department of Emergency Medicine, an Associate Professor and Tier-2 Research Chair in Competency-Based Medical Education (CBME), and a Clinician Educator Senior Consultant for the Centre for Innovation in Medical Education (CiMED), at the University of Ottawa in Canada. In addition, he is a Clinician Educator focused on the evaluation of the Competency-By-Design (CBD) initiative at the Royal College of Physicians and Surgeons of Canada.
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa . For more details on our site disclaimers, please see our ‘About’ page