Hallucination filtering in radiology vision-language models using discrete semantic entropy

Clinical Snapshot

Key Findings

• P Value: p < 0.001 for both models at DSE >0.3 threshold; Bonferroni-corrected significance threshold p < 0.004 applied for multiple comparisons

• Effect Size: GPT-4o: baseline accuracy 51.7% → 76.3% after DSE >0.3 filtering (absolute improvement +24.6 percentage points; 47.6% relative improvement, retaining 334/706 questions, 47.3% coverage). GPT-4.1: baseline accuracy 54.8% → 63.8% after DSE >0.3 filtering (absolute improvement +9.0 percentage points; 16.4% relative improvement, retaining 499/706 questions, 70.7% coverage)

• Primary Outcome: Diagnostic answer accuracy on retained image-question pairs after DSE-based hallucination filtering compared to unfiltered baseline accuracy across 706 image-question pairs from two radiology datasets

• Nnt Or Sensitivity: Coverage-accuracy trade-off: at DSE >0.3 threshold, GPT-4o answers only 47.3% of questions (334/706) to achieve 76.3% accuracy; GPT-4.1 answers 70.7% (499/706) to achieve 63.8% accuracy. Sensitivity and specificity of DSE as a hallucination detector against a clinical reference standard are not reported

• Confidence Interval: 95% CIs reported via bootstrap resampling; specific interval values not provided in abstract — reported as statistically significant and largely surviving Bonferroni correction

Clinical Application

DSE computation requires 15 repeated API calls per question at temperature 1.0, substantially increasing computational cost and latency compared to single-pass inference. For a 706-question dataset, this represents 10,590 API calls. In real-time clinical workflows, this latency and cost may be prohibitive. The method is model-agnostic (black-box) and does not require access to model internals, which is a practical advantage for commercial API-based tools. Implementation requires an NLI (natural language inference) model for entailment checking, adding infrastructure complexity. No Australian-specific radiology datasets or clinical workflows were evaluated. In the Australian context: (1) GPT-4o and GPT-4.1 are not TGA-approved as medical devices for diagnostic use, and their deployment in clinical decision support would require TGA Software as a Medical Device (SaMD) regulatory consideration; (2) RACGP and RANZCR (Royal Australian and New Zealand College of Radiologists) have not endorsed VLM-based diagnostic tools, and RANZCR's AI standards framework would require prospective clinical validation before adoption; (3) Medicare Benefits Schedule (MBS) does not currently reimburse AI-assisted radiology interpretation as a standalone service; (4) Australian radiology practice involves significant rural and remote teleradiology contexts where AI decision support tools may have particular utility, but this study's benchmark datasets do not reflect this case mix; (5) Privacy considerations under the Australian Privacy Act 1988 and My Health Record Act 2012 would govern any deployment of patient imaging data through commercial API services Radiologists and clinical AI developers considering deployment of commercial vision-language models (specifically GPT-4o or GPT-4.1) for image-based question answering in diagnostic radiology settings. Most directly applicable to structured VQA tasks (e.g., decision support tools, teaching platforms) rather than free-text report generation

Abstract

OBJECTIVE: To determine whether using discrete semantic entropy (DSE) to reject questions likely to generate hallucinations can improve the accuracy of black-box vision-language models (VLMs) in radiologic image-based visual question answering (VQA). MATERIALS AND METHODS: This retrospective study evaluated DSE using two publicly available, de-identified datasets: the VQA-Med 2019 benchmark (500 images with clinical questions and short-text answers) and a diagnostic radiology dataset (206 cases: 60 computed tomography scans, 60 magnetic resonance images, 60 radiographs, 26 angiograms) with corresponding ground-truth diagnoses. GPT-4o and GPT-4.1 (Generative Pretrained Transformer) answered each question 15 times using a temperature of 1.0. Baseline accuracy was determined using low-temperature answers (0.1). Meaning-equivalent responses were grouped using bidirectional entailment checks, and DSE was computed from the relative frequencies of the resulting semantic clusters. Accuracy was recalculated after excluding questions with DSE > 0.6 or > 0.3. p values and 95% confidence intervals were obtained using bootstrap resampling and a Bonferroni-corrected threshold of p < 0.004 for statistical significance. RESULTS: Across 706 image-question pairs, baseline accuracy was 51.7% for GPT-4o and 54.8% for GPT-4.1. After filtering out high-entropy questions (DSE > 0.3), accuracy on the remaining questions was 76.3% (retained questions: 334/706) for GPT-4o and 63.8% (retained questions: 499/706) for GPT-4.1 (both p < 0.001). Accuracy gains were observed across both datasets and largely remained statistically significant after Bonferroni correction. CONCLUSION: DSE enables reliable hallucination detection in black-box VLMs by quantifying semantic inconsistency. This method significantly improves diagnostic answer accuracy and offers a filtering strategy for clinical VLM applications. KEY POINTS: Question Can DSE identify hallucination-prone questions and improve the reliability of black-box vision-language models in radiologic image-based VQA? Findings DSE filtering at a 0.3 threshold increased GPT-4o accuracy from 51.7% to 76.3% and GPT-4.1 from 54.8% to 63.8%, while answering fewer questions. Clinical relevance Integrating DSE as a black-box uncertainty filter enables selective answering and explicit uncertainty display for radiology vision-language tools, supporting safer diagnostic use, mitigating hallucinations, and improving clinicians' trust in AI-assisted image interpretation.