By: Victoria Brazil (@SocraticEM)
How might the emerging capabilities of data science, with AI, help our healthcare simulation practice? There are many published examples of using Chat GPT to write scenarios – faster and (nearly) as well as experienced simulation practitioners. AI tools can help us generate ECGs, images and blood gas results ‘on demand’ to support those scenarios. One can even use Co-pilot to edit blog posts about AI and simulation (😊).
But what about deeper applications of data science…?
What do we mean ‘data science’..?
In simple terms, it’s the art and science of extracting meaningful insights from data. Think of it as a treasure hunt: you’ve got a map (data), a shovel (algorithms), and a treasure chest full of valuable insights waiting to be uncovered. Whether it’s predictive analytics, machine learning, or big data analysis, data science is the treasure hunter of the modern age.
(these prior 3 sentences were written by Chat GPT…)
Data informed simulation design
Educational needs analysis can now draw upon ‘big data’, i.e. analytics describing the current performance of individual learners (and the cohorts they are part of) in a range of clinical, relational and behavioral skills. For example, Thau et al. found that “Using learning analytics, we identified actionable learning opportunities for paediatric CXR interpretation, which can be used to allow for a customized weighting of which cases to practice.” Lots of educational platforms have embraced this customized learning approach – DuoLingo, Qstream and many others. Procedural skills are at the vanguard of this work in health professions education, as they have well-established quantitative performance metrics and learning curves. But with improved ability to collect and analyze narrative data, these principles can be applied to simulation design for communication, teamwork and behavioral skills. The potential to use real world performance for this purpose is waiting to be realized but limited by the current poor quality of data collection and reporting in clinical EMRs.
For translational simulation – simulation focused on quality improvement and organizational learning – AI can help problem definition and simulation planning. Bespoke large language models (LLMs) can be trained to draw upon focused resources and structured prompts to design simulation strategies. A query like “design a simulation strategy to support operational readiness for a team moving to a new hospital” would be answered with a detailed approach to stakeholder engagement, sample scenarios, advice on appropriate simulation modalities, debriefing guidance and approach to data collection and analysis. As with most AI, such a strategy still requires expert oversight and adjustment to context, but it’s a great starting point.
Data informed debriefing and feedback
This application of data science already has many exemplars. It will be more readily available to the everyday simulation practitioner as data collection and analysis tools move to consumer level technology.
The Laerdal RQI product* is an obvious example for a relatively simple task. The machine gives accurate real time feedback on compression rate and depth, saving valuable instructor time (and avoiding their fallibility as assessors). This can also work for teams. Andrew Petrosoniak described his approach to data integrated simulation in this talk/ podcast, using CPR performance data in simulation and in the real world to support feedback and improvement.
Data informed debriefing can have clinical impact. Vadla et al showed an increase in newborns ventilated within the first minute of life and reduced mortality after clinical data-guided simulation training. The challenge is deciding what and how to collect that data, and how to analyze it and present it usefully to healthcare professionals and teams.
More complex tasks are now within this remit. Twenty years ago, I would have said patient communication was a nuanced, human task that was more art than science. But industry leaders like Sim Converse have used AI to inform their avatar-based communication training, with granular and actionable feedback from the ‘machine’.
Actionable outcomes from simulation debriefings
But the real opportunity for translational simulation might be what we get out of the debrief for organizational learning/ QI purposes, i.e. the participant perspectives on the strengths and deficiencies of the systems in which they work. Thanks to improved AI tools, anyone can record a debrief with a mobile phone and use tools to analyze and summaries these conversations for themes and action points (think your Zoom or Teams meeting summaries). These can be aggregated and provide a helpful input into quality improvement ‘system sensing’. This could offer a marked improvement on the current situation where so much wisdom is often ‘left in the room’…
Where to from here…?
So, perhaps the take home message is that our simulation design, delivery and debriefing should have a data strategy – whether simple or complex – enabled by emerging technology, tools, and AI applications.
Happy simulating!
Victoria
NB. *Disclosure – I have worked with Laerdal Medical. I have no disclosures with other products mentioned
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The University of Ottawa. For more details on our site disclaimers, please see our ‘About’ page