Home › Operations & systems › Protocol-grounded LLM gave 75% accurate advice

Operations & systems

Protocol-grounded LLM gave 75% accurate advice

Can a Large Language Model Grounded in Text-Based Agency-Specific Prehospital Protocols Provide Accurate Care Recommendations?
Prehospital Emergency Care · Added 11 May 2026

This study tested whether a large language model trained on agency-specific prehospital protocol documents could generate accurate care recommendations — they found promising accuracy for protocol interpretation and clinical decision support, suggesting potential for LLMs to assist with guideline navigation at the point of care.

Study type

Simulation

Design

Non-human simulation-based experimental evaluation of a RAG-based LLM

Participants

6 clinical scenarios; 169 expected patient care actions

Main finding

The LLM recommended 127 of 169 actions (75%); 5% were 'major misses' (5 of 9 in paediatric OHCA), 8% minor misses, 12% non-applicable; 12 hallucinations identified, none judged to endanger patient safety.

Caveat

Only six scenarios in a single EMS agency; no clinical use or patient outcomes tested

Bottom line

Hypothesis-generating

Learning goal

Understand the risks and safeguards when using AI decision support tools to navigate prehospital protocols at the point of care.

More

• Topic: Operations & systems
• Daily scan: 11 May 2026

Related papers on Operations & systems

• Medical planning for 2024 Republican convention

  Prehospital Emergency Care · 15 May 2026 · Case series · Context only

  This case report describes comprehensive health and medical planning and multi-agency response for a large-scale national political convention, providing a template for mass gathering EMS preparedness.

• Critical care attendance driven by distance, not deprivation

  Resuscitation · 14 May 2026 · Cross-sectional · Context only

  This spatial analysis examined how deprivation levels and prehospital critical care service configuration affect cardiac arrest outcomes, revealing persistent geographic disparities in access and survival.

• Paramedic treat-and-discharge rarely used but efficient

  Prehospital Emergency Care · 11 May 2026 · Cohort · Context only

  This retrospective cohort study analysed paramedic treat and discharge medical directives to understand utilisation patterns and key metrics for prehospital and emergency department decision-making — the findings provide valuable insights into the scope, safety, and operational impact of treat-and-refer pathways.

• Eye emergency calls often chemical or stroke

  Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine · 9 May 2026 · Registry · Context only

  Register-based cohort of 340 eye-related emergency calls found chemical exposures (31.5%), vision loss (25.9%), and trauma (18.2%) as leading causes, with 11.5% of patients initially coded for eye problems later diagnosed with stroke.