Omission and hallucination prevalence of clinical guidelines in diagnostic large language model outputs

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: Omission rates: up to 97% (DeepSeek-V3) and 46% (GPT-4.1); Hallucination rates: up to 9%

• Primary Outcome: Prevalence of clinical guideline omissions and hallucinations in LLM diagnostic outputs

• Nnt Or Sensitivity: Guideline citation rate variability: 0% to 78.39% across sociodemographic vignettes

• Confidence Interval: Not reported in abstract

Clinical Application

Results suggest current LLMs require significant oversight and validation before clinical deployment Highly relevant to Australian healthcare given TGA considerations for AI medical devices and RACGP digital health guidelines; supports cautious approach to LLM implementation Healthcare systems and clinicians considering implementation of LLMs for diagnostic support

Abstract

OBJECTIVE: Meaningful assessments of how large language models (LLMs) incorporate clinical guidelines require large-scale testing over many queries. Here, we evaluate the prevalence of clinical guideline omissions and hallucinations in a large sample of diagnostic LLM outputs. METHODS: We used simulated case vignettes and zero-shot prompting to generate diagnostic outputs and rationales from GPT-4.1 and DeepSeek-V3. English case vignettes were created for hypercholesterolaemia and type-2 diabetes mellitus. Each vignette contained identical medical information, while sociodemographic characteristics varied in terms of sex, ethnicity and location. We calculated the prevalence of existing and hallucinated clinical guidelines in LLM outputs across disease, LLM and sociodemographic characteristics. RESULTS: We analysed a total of 12 197 LLM outputs, which quantifies three hazard areas: omissions (up to 97% for DeepSeek-V3 and 46% for GPT-4.1), hallucinations (up to 9%) and inconsistencies (guideline citation rate ranging from 0% to 78.39% across sociodemographic vignettes). Omission and hallucination rates were generally similar across vignettes with different sex or ethnicity data, yet were particularly sensitive to patient location. DISCUSSION: This study highlights significant variability in clinical guideline prediction across two different diseases, three different sociodemographic variables and two LLMs, even when the LLMs were instructed by identical prompts, establishing clinical guideline prediction in LLM outputs as a stochastic event. CONCLUSION: The stochastic nature of LLMs creates a unique challenge for evidence generation and clinical deployment. Being able to measure and capture this stochasticity within high-quality research designs will be a prerequisite to advancing the responsible deployment of LLMs in healthcare.

References

1. 1.van Kessel, R., Anderson, M., McMillan, B., Matthews, M. R., Rust, P., Pearcy, P., Nasir, K., & Mossialos, E. (2026). Omission and hallucination prevalence of clinical guidelines in diagnostic large language model outputs. BMJ Health & Care Informatics, 33(1), e101959. https://doi.org/10.1136/bmjhci-2025-101959