#KeyLIMEpodcast 138: The Gender Gap in Direct Observation Assessment

Gender bias in the workplace should come as no surprise to most.  Yet, many do not realize how pervasive sexism can be in the assessment of trainees in our residency programs. This week’s selection showcases the significant issue of gender bias in the academic environment and the hosts attempt to understand the scope of the problem.

What have been your experiences regarding gender bias? What do the hosts think? Check out the podcast here (or on iTunes!) to find out!


KeyLIME Session 138 – Article under review:

Listen to the podcast

View/download the abstract here.


Dayal A, O’Connor DM, Qadri U, Arora VM. Comparison of Male vs Female Resident Milestone Evaluations by Faculty During Emergency Medicine Residency Training. JAMA Internal Medicine. 2017. March 6. [ePub ahead of print]

Reviewer: Jon Sherbino (@sherbino)                                          


Gender bias in the workplace should come as no surprise to KeyLIME-ers.  Yet, I must confess that I (naively) assume that sexism is not in play in the assessment of trainees in our residency programs. Presumably, a systematic approach to assessment, the enrollment of women in medical school exceeding 50% and the active recruitment of women leaders into academic medicine should counter established and insidious patterns in society.  This paper challenges my assumption.  I am wrong.  This is a significant problem.  And sadly this is not the first paper to demonstrate this issue.  A recent survey of American academic faculty found that >75% women perceived gender bias in the academic environment.  So, let’s open our eyes, move past assumptions, and understand the scope of the problem.


This study aims to compare the evaluation of male vs female residents by faculty throughout training using a novel longitudinal, multi-institutional data set that consists of EM milestone evaluations based on direct observation.”

Type of Paper

Research: Retrospective cohort

Key Points on Methods

This study received REB exemption.

8 hospital-based residency training programs contributed two years of direct observation, real-time data via a mobile app for milestone assessments.  All programs were three-year ACGME-accredited emergency medicine residency training programs.

Observation was at the discretion of faculty, typically completing one to three observations per shift.  One of 23 EM competencies was scored using a 5-point behaviourally-anchored scale and optional free text.

Data was analyzed as continuous (not ordinal) data.   A 3-level mixed-effects linear regression model was used.  Confounders addressed in the model included: community or academic based program, the gender of the faculty, and interaction between faculty and resident gender.

Key Outcomes

33,456 observations of 359 residents (66% male) by 285 faculty (68% male) were included in the analysis.

Training programs represented all 4 US census-designated regions.

The mean number of assessments per resident was 96 for female residents and 87 for male residents, which was not significantly different.  The number of observations by faculty was 101 for female faculty and 125 for male faculty, again not significantly different.

There was no significant difference in overall milestone achievement based on gender in the first year of training. In fact, women received higher mean milestone scores on 8 of 23 competencies in PGY 1.

There was no significant difference between genders in training gaps, suggesting that maternity leave was not a significant confounder.

By the third year of training, men achieved a higher milestone level in all 23 competencies.  This was consistent across 7 of 8 programs.  There was no statistically different scoring of men depending on the gender of the faculty observer.

The linear regression model indicated that men had a 13% higher rate of milestone achievement. This is equivalent to a three to four month additional observation period required for women to achieve the same milestone level.

There was no effect on the model based on the gender of the faculty observer or resident–faculty gender interaction.

Key Conclusions

The authors conclude…

Although male and female residents receive similar evaluations at the beginning of residency, the rate of milestone attainment throughout training was higher for male than female residents across all EM subcompetencies, leading to a gender gap in evaluations that continues until graduation.”

Shout out

Shout out to feminem.org, a grass roots organization started by Daria Kass to support women in academic emergency medicine.  FeminEM flagged this paper for me and led an informative on-line discussion about the findings and implications of the study.

Shout out to Vinny Arora, the senior author of this paper, who mentored medical student co-authors, including the first author for this paper.

Access KeyLIME podcast archives here

Check us out on iTunes