Context…Performance…Recommendation and Reinforcement (CPR2): Bringing supervisor narrative comments to life in Competency Based Medical Education

By: Rob Woods (@robwoodsuofs) and Jeff Elder (@JeffreyDElder)

Vignette:

You arrive for work (shift, clinic, OR) a few minutes early knowing you have a learner with you. You sit down with the trainee, set up expectations for the day, try to identify any goals they have and determine what assessments or forms that need to be filled out. The trainee is a second-year resident and they indicate they are short on specific types of cases and were hoping to target that workplace-based assessment (WBA) if it comes up (Entrustable Professional Activity aka EPA or Field Note). If not, their back up WBA is anything where point of care ultrasound will be used. You embark on your clinical work and both opportunities arise. You are able to observe portions of the encounter and have productive coaching conversations about each case. You are ending your day of work finishing off your charts and find the email notification to fill out those two assessment forms…now what should you write down?

Coaching in the Moment with narrative comments…why do they matter?

With the adoption of Competency Based Medical Education (CBME) in Canadian postgraduate medical training, there is heavy emphasis on WBAs.^1,2 Seeing performance in practice sits at the top of Miller’s pyramid of assessing clinical competence, but it requires at least some of the encounter to be directly observed.³ Educators have moved to use the language of coaching for this form of assessment, broken down into coaching in the moment (as in the vignette above) and coaching over time, where a faculty coach or Academic Advisor meets with trainees at specified intervals to help establish learning goals. Approaches to coaching in the moment in CBME have been developed. The Rapport Expectations Observe Coach Record (RX-OCR) model has been promoted by the Royal College of Physicians and Surgeons of Canada as one potential approach (https://www.royalcollege.ca/en/cbd/impact-cbd/coaching-and-cbd.html). The last ‘R’ in this model is to ‘Record’ the coaching conversation. Front line faculty are often frustrated by this, as they have worked hard to incorporate teaching and now direct observation into their clinical workload, but filling out assessment forms can feel redundant and time consuming.^4,5 I mean, you already had the conversation, right?

It is important for front line faculty coaches to understand how these recorded narrative comments get used in the ‘back-end’ of CBME.^6,7 Simply ticking some demographic and context tick boxes⁸ then writing ‘good job’ may be enough for the trainee to remember the conversation, but to the Academic Advisor or Competence Committee member, this will mean very little.⁹ The literature is telling us that narrative comments are key when it comes to assessment.¹⁰ While anything in assessment is far from perfect, narrative comments hold a lot of potential.^11,12 Once front-line faculty coaches know that these narrative comments are read and used by those who were not there in the moment, they can be better motivated to write more detailed information.

So narrative comments are important, but it’s not easy!

Providing critical feedback to trainees can be uncomfortable.¹³ We worry that the trainee may take the feedback poorly. We don’t see our colleagues do it, so why should we? We worry that the consequences may be too harsh; what if this was just an off day? As a result, we engage in some ineffective behaviours: staying quiet, doing a ‘learner handover’ and hedging.^14–16 We have all let a learner’s poor performance go unsaid at times for the above reasons, but this is not helping the trainee. They go through day after day not hearing anything critical and assume they are doing great. Sometimes we do a learner handover and tell our colleague who is working with them next, hoping they will have the confidence to say something if they know it’s more than a one-off poor performance. Once again, this is not helping the learner. Then if we do have the courage to document something we exaggerate approval and hedge. For below average performers we use ‘good’ or ‘solid’. Then when learners are actually meeting expectations, we use ‘outstanding’ and ‘excellent’. If the trainee did not do well today, we offer promise: ‘I am confident the trainee will do it well next time’. While this may be good for morale in your trainee, it does not help the trainee or the program in the back help to develop a specific learning plan of how to get better.¹⁷

How can I simplify this?

Efforts have been made to identify what constitutes a high-quality supervisor comment.^18,19 The Quality Assessment for Learning (QuAL)²⁰ and Evaluation of Feedback Captured Tool (EFeCT)²¹ have been developed specifically for WBAs in the Canadian context of competency based medical education in Royal College Specialty and Family Medicine training programs respectively. Both tools identify elements that make a comment useful to the end-users of this data. This includes not only the trainees receiving it, but their longitudinal coaches who help them interpret aggregate data over time and competence committee members and periodic reviewers who are in charge of entrustment, progress decisions and learning plans.²² Both of these tools were designed to measure a comment after it has been created, but we can take the key elements of this tool and turn into a framework or guide for faculty to craft a useful narrative comment (Table 1).

Table 1 – The CPR²Approach to a High-Quality Narrative Comment

Approach to a High-Quality Narrative Comment
QuAL elements	CPR²	EFeCT elements
Was there adequate description of performance?	Context? Demographics, co-morbidities, ultimate diagnosis	Context?
	Performance? What did the learner do? What did you have to help them with?	What did the learner do?
		How did the learner do?
Was there a recommendation for improvement? Were the description and recommendation linked?	Recommendation and/or Reinforcement? What can they do better next time? What should they keep doing? Be specific.	What was done well or needs improvement?
		How was it done well or how can it be improved?

Now while WBAs are not the only tools in a program of assessment, they are the dominant tool so we should strive to achieve a high standard of quality. Each WBA serves as a low stakes in the moment formative assessment, but they are used collectively in a summative fashion by the Academic Advisor for coaching over time and by the Competence Committee for progress decisions.⁷ If the data from WBAs is of high quality, the back-end of CBME can provide useful information to a trainee to inform their goals and learning plans. If the data is poor, the trainee can feel lost. (Figure 1) With a little practice, the CPR² framework can dramatically improve the quality of a narrative comment. (Table 2)

Table 2 – Sample Narrative Comments

Sample Narrative Comments	How can we make this better?	Improved with CPR²
45 y o with abdo pain, good history, keep reading around cases	Context: needs more detail on co-morbidities, acuity, ultimate diagnosis Performance: unclear what the resident needed help with Recommendation/Reinforcement: needs to be more specific/actionable	The trainee assessed a previously healthy 45 y o male who was ultimately diagnosed with diverticulitis. They assessed the patient and ordered investigations without help; I provided guidance around therapy and disposition. In future cases, use shared decision making around the use of antibiotics for uncomplicated diverticulitis.
3 y o female with pneumonia, needed help with CXR interpretation	Context: add more detail about presenting symptoms, co-morbidities Performance: good job of clearly identifying what they needed help with Recommendation/Reinforcement: be more specific about how to improve their CXR interpretation. Could add what they did well	The trainee assessed a previously healthy 3 y o female with fever for > 5 days, who was diagnosed with occult pneumonia. The trainee appropriately wanted to work up fever > 5 days for occult pneumonia, but needed help interpreting the CXR. You specifically looked for symptoms of Kawasaki’s in this child, which is very important in pediatric patients with prolonged fever; keep this up. With CXRs, use your PA and lateral xrays together to localize lobar air space disease: RML/lingula – right heart border on PA/anterior chest over heart on lateral, RLL/LLL – diaphragm/retro-cardiac on PA, posterior base on lateral.
22 y o female with laceration to arm, work on suture technique	Context: how did they get the laceration? Performance: how big was the laceration, what techniques were used, what did they need help with? Recommendation/Reinforcement: good job suggesting they need to work suturing, but try to be more specific	A 22 y o female presented with a large forearm laceration after falling into a glass window. The trainee screened for intimate partner violence, assessed the wound for foreign bodies, as well as a tendon function and sensory/motor exam. I needed to prompt them to use a 2-layer closure for tension. Good job screening for intimate partner violence in this case; it is very common with females who present to the ED with any type of trauma. With deep wounds, using a 2-layer closure helps to eliminate a potential space for hematoma/infection and reduces tension on the surface to mitigate risk of dehiscence.

Not only are we guiding the trainee during their training²³ to meet the objectives of their specialty, we are also trying to set them up for success in the skills of life-long learning. Medicine is a self-regulating profession, so part of the goals of our training programs should be to train physicians to do so. Self-regulated learning theory²⁴ involves trainees setting goals and using regulatory mechanisms to meet these goals. It requires some external support to help with monitoring of performance and metacognition. Follow up regulatory appraisals require further self-evaluation, attributions for performance and self-efficacy. Once trainees graduate, they will not have as much external data to guide their performance. It is important that we not only teach trainees how to do their clinical work, but also how to set them up with skills to inform their life-long learning.

Figure 1 – The Importance of High Quality Supervisor Narrative Comments in CBME

So next time you have a WBA to fill out, think CPR². Take that dead ‘good job, keep reading’ comment and bring it to life by describing the context, performance, recommendation and/or reinforcement for that case. Useful narratives are critical for the learner AND the training program to help the trainee make an effective learning plan to improve.

About the authors:

Rob Woods, MD MMEd FRCPC DRCPSC, is a Professor of Emergency Medicine at the University of Saskatchewan in Saskatoon. He works in the Adult & Pediatric Emergency Departments and as a Transport Physician with STARS Air Ambulance. He served a 13-year term as Program Director for Emergency Medicine. He currently serves as the lead for the Clinician Educator Diploma Program, co-Research Director for Emergency Medicine and a Decision Editor for the Canadian Journal of Emergency Medicine. He current research focus is in feedback and assessment in CBME. Check out www.commentquality.ca.

Jeff Elder, BSc. MD is a PGY-4 in the FRCP Emergency Medicine program at the University of Saskatchewan. He is currently enrolled in the clinical educator diploma program and his interests include the integration of technology and medical education.

Context…Performance…Recommendation and Reinforcement (CPR2): Bringing supervisor narrative comments to life in Competency Based Medical Education

Vignette:

Coaching in the Moment with narrative comments…why do they matter?

So narrative comments are important, but it’s not easy!

How can I simplify this?

About the authors:

Jeff Elder, BSc. MD is a PGY-4 in the FRCP Emergency Medicine program at the University of Saskatchewan. He is currently enrolled in the clinical educator diploma program and his interests include the integration of technology and medical education.

References

Like this:

Vignette:

Coaching in the Moment with narrative comments…why do they matter?

So narrative comments are important, but it’s not easy!

How can I simplify this?

About the authors:

Jeff Elder, BSc. MD is a PGY-4 in the FRCP Emergency Medicine program at the University of Saskatchewan. He is currently enrolled in the clinical educator diploma program and his interests include the integration of technology and medical education.

References

Share this:

Like this:

Discover more from ICE Blog