Improving Trauma Triage Accuracy with Large Language Models: A Comparison to Human Expert Decisions

Clinical Snapshot

Key Findings

• P Value: p < 0.01 for accuracy comparison (Bonferroni corrected p < 0.01)

• Effect Size: LLM triage accuracy 83.5% vs human 78.9% (4.6% absolute improvement)

• Primary Outcome: LLM undertriage rate 4.8% vs human 5.1% (non-significant difference)

• Nnt Or Sensitivity: Number needed to treat with LLM assistance: approximately 22 cases to prevent one triage error

• Confidence Interval: LLM accuracy 80.6-90.6% vs human 73.9-82.6%

Clinical Application

Requires significant technological infrastructure including audio transcription, LLM integration, and staff training for implementation Would require adaptation to Australian emergency medical service protocols, integration with state-based trauma systems, and consideration of TGA regulatory requirements for clinical decision support tools Pediatric trauma patients in emergency medical service systems with audio recording capabilities

Abstract

BACKGROUND: Accurate prehospital trauma triage and communication determine morbidity, mortality, and system efficiency. Advancements in large language models (LLMs) offer an opportunity to improve triage and yet to be implemented in prehospital trauma triage. STUDY DESIGN: This retrospective cohort study evaluates LLM performance in trauma triage and accuracy of prehospital tele-communication. Of 410 pediatric activations at a Level I center (January 2023 to May 2025, IRB no. 00009569), 133 activations with emergency medical service recordings, human-generated trauma pages, and injury severity scores were analyzed. Audio was transcribed with OpenAI Whisper. Structured "Essential Transcripts" were generated with named entity recognition. Entity ablation tested redaction of triage parameters on accuracy. In a prospective arm, trauma surgeons reviewed emergency medical service transcripts and triaged activations pre- and post-LLM exposure. Cribari criteria defined over and undertriage. McNemar's test and 95% CIs assessed paired differences in accuracy. RESULTS: The primary endpoint: LLM undertriage demonstrated modest improvement; 4.8% (3 to 8.2) vs 5.1% (3.1 to 9.3, p = 0.73, Bonferroni p = 1). For secondary endpoints, LLM triage outperformed human clinicians: 83.5% accuracy (80.6 to 90.6) vs 78.9% (73.9 to 82.6, p < 0.01, Bonferroni p < 0.01), with overtriage 58.6% (51.4 to 73.7) vs 71.8% (p < 0.05, Bonferroni p < 0.09). "Essential Transcripts" reduced transcript length by 80.8% (p < 0.001) while preserving accuracy (81.9%, 76.6 to 87.5, p < 0.001, Bonferroni p < 0.05). Entity ablation had marginal effect on triage. In prospective evaluation, human triage accuracy improved after LLM exposure (73.7% [69.8 to 77.2]) to 75.8% ([71.9 to 79.2], p = 0.04, Bonferroni p = 0.12), significantly improving the odds of a correct triage decision (odds ratio 2.57, 95% CI 1.39 to 6.83, p < 0.01). CONCLUSIONS: LLMs achieve triage accuracy comparable to trauma staff in retrospective review of pediatric trauma. Further validation is needed to assess clinical outcomes, generalizability, and user acceptance before widespread deployment.

References

1. 1.Balaji, A. K., Fox, B. T., Seger, P., Gorugantu, A., Nordin, A., Fabiano, T., Yang, G., Himidan, S., Schwaitzberg, S. D., & Kim, P. C. (2026). Improving trauma triage accuracy with large language models: A comparison to human expert decisions. Journal of the American College of Surgeons. https://doi.org/10.1097/XCS.0000000000001895