Negotiations in CCCs to arrive at consensus about summative entrustment: would a Habermas Machine be able to assist?

By: Olle ten Cate, PhD

October 18, 2024, Science published a UK study showing how an AI tool (the Habermas Machine) outperforms a human mediator in producing consensus after debates among groups with divergent political convictions about controversial issues (e.g., ‘should the UK apply to rejoin the European Union?’)(1). Philosopher Habermas’ Theory of Communicative Action posits that when rational people deliberate under idealized conditions, agreement will emerge in the public sphere.

The authors state that human society is enriched by a plurality of legitimate, but divergent viewpoints, and agreement is a prerequisite for people to act collectively. Mediation is often needed to reach consensus in such situations.

Frankly, I knew little about Habermas’ work, but triggered by the Science paper I tried to understand his thinking, if even superficially. Habermas’ focus is on three conditions in any critical discourse debaters should qualify for: true (does a debater state the truth?), normative (is a debater qualified to judge?), and sincere (does a debater refrain from a hidden agenda?).

The Habermas Machine, a large language model, was trained to find common ground among a group of people discussing a social or political issue, and has the capacity to iteratively revise a group statement by incorporating written critiques from the individual group members through the same generation and selection process. It is thus designed to arbitrate among the various views by private interaction with each discussant.

The idea is fascinating and I started wondering if something similar could help in negotiations within clinical competency committees. A reasonable assumption is that CCC members meet the three Habermas conditions. These committees are charged with the task to reach consensus about decisions regarding trainees. In the philosophy of programmatic assessment, datapoints from multiples sources of workplace-based assessment are to be aggregated and reviewed for high stakes decisions, which could either regard progress of learners (2) or on prospective, summative entrustment decisions to allow for more autonomy in patient care (3) or both.

The information to ground high stakes decisions (eventually, for completion of training and readiness for practice) is potentially overwhelming. Workplace-based assessment information includes direct observations, case-based discussions, longitudinal observations (such as multisource feedback) and evaluation of products of care delivered.(4) Traditional technology can help to synthesize data in accessible dashboards of e-portfolio data(5), but members of clinical competency committees may still have legitimate information that is not easily captured in such data (6,7).  Negotiations in CCC meetings are basically consensus processes that need to optimize the validity of decisions. While these to not regard politics as in the Science study, they are high stakes. I have facilitated many mock CCC meetings in our international EPA course. More recently, I’ve had the opportunity to observe a few actual CCC meetings as a guest. This left me, while impressed with the rigor that was aimed at, with the feeling that divergent opinions exists and that often members may have difficulty to oversee the multitude of data. So I wondered: might AI help to arrive at optimized summative entrustment decisions when the committee members have divergent views?  As narrative information is being increasingly recommended to weigh more heavily than numbers on assessment scales, and as more than specific knowledge and skill should be incorporated(8), committees may benefit from support to help see the forest from the trees in rational decision making.

It might be worth considering creating a CCC Habermas Machine by training it on portfolio data from workplace-based assessments and from recorded negotiations in CCC meetings, to help suggest consensus decisions that the committee can choose to adopt.

Of course, CCC members are not individuals with opposing political convictions that require mediation to come to consensus policy. In addition, the inclusion of portfolio data in an adapted Habermas Machine would go beyond individual opinions.

The Machine should be constructed in a way to support the decision-making when all feel their views may diverge too much. Its use may range from a having access to all possible information, and comes with a proposal, to one that is limited to mediate when consensus is difficult in an ed stage. Food for thought.

References

1. Tessler MH, Bakker MA, Jarrett D, Sheahan H, Chadwick MJ, Koster R, et al. AI can help humans find common ground in democratic deliberation. Science. 2024 Oct 18;386(6719):eadq2852.
2. van der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. 12 Tips for programmatic assessment. Med Teach. 2014 Nov 20;37(7):1–6.
3. Touchie C, Kinnear B, Schumacher D, Caretta-Weyer H, Hamstra SJ, Hart D, et al. On the validity of summative entrustment decisions. Med Teach. 2021 Jul;43(7):780–7.
4. ten Cate O, Jonker G, Park YS, Holmboe ES, Burch VC. Workplace-based assessment to support entrustment decision-making: four sources of information. In: ten Cate O, Burch VC, Chen HC, Chou FC, Hennus MP, editors. Entrustable Professional Activities and Entrustment Decision-Making in Health Professions Education. 1st ed. London: Ubiquity Press; 2024. p. 197–211.
5. Marty AP, Linsenmeyer M, George B, Young JQ, Breckwoldt J, Ten Cate O. Mobile technologies to support workplace-based assessment for entrustment decisions: Guidelines for programs and educators: AMEE Guide No. 154. Med Teach. 2023 Nov;45(11):1203–13.
6. van Enk A, Ten Cate O. “Languaging” tacit judgment in formal postgraduate assessment: the documentation of ad hoc and summative entrustment decisions. Perspect Med Educ. 2020 Dec;9(6):373–8.
7. van Enk A, MacDonald G, Hatala R, Gingerich A, Tam J. Not in the file: How competency committees work with undocumented contributions. Med Educ. 2024 Nov;58(11):1333–42.
8. ten Cate O, Chen HC. The ingredients of a rich entrustment decision. Med Teach. 2020 Dec;42(12):1413–20.

The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa . For more details on our site disclaimers, please see our ‘About’ page