Prioritizing Safety and Efficiency With Machine Learning
Northwestern Medicine Data Scientists Create Tool to More Efficiently Monitor Safety
A culture of safety supports clinicians and patients alike and ensures clinicians deliver consistent, high-quality care.
Event-reporting systems require health systems to monitor patient safety events and unsafe conditions. These systems generate a high volume of unstructured risk and safety data that require manual effort to analyze and prioritize. At Northwestern Medicine, the Risk Management and Workforce Health & Safety team review these cases and take the appropriate steps to address and resolve them.
Efficient workflows can help teams review these reports quickly, prioritize high-risk events and escalate specific concerns. That’s why the data scientists at Northwestern Medicine created Safety AI Reporting (SAfeR), a large language model (LLM) to organize and triage high-risk events to more efficiently monitor patient safety.
Making a SAfeR Environment for All
The engineered prompts are:
- Did the report involve violence against the clinicians, and if so, is it verbal or physical?
- Did it involve medication errors with Heparin, an anticoagulant taken to prevent blood clots, or insulin?
- Did the report involve medication dosing errors?
By pulling answers to these prompts from the reports, the tool creates a summary, including a binary flag for the presence of certain items. The Risk and Safety Team can then review the reports and triage them based on priority.
The model is performing exceptionally well, with more than 89% agreement with human review.
Improving Efficiency and Clinical Decision-Making
One of the benefits of the SAfeR solution is the operational efficiency.
+200 reports
are processed per day
~84 hours
of review time saved per day (vs manual review)
This reduced time helps with timely intervention
It can also detect patterns to prevent issues from occurring. SAfeR enables operational leaders to view trends in data across location, department and team level. For example, this can help identify departments with a higher rate of medication errors rates, allowing leaders to prioritize training and process review. SAfeR can also show if there are certain hospitals or units that experience safety events more often than average to create an appropriate plan of action to mitigate.
The tool helps identify patients who may be at a higher risk of harming themselves or others, based on reported events. The patient may be referred to the Patient Behavioral Risk Review Committee for appropriate safety interventions.
SAfeR continues to evolve since its implementation. The team has started combining the output with other medical record data to help the Quality and Patient Safety team assess the tool’s impact.
Discover other ways LLMs are impacting patient care at Northwestern Medicine.
