Large language models for optimizing clinical trial recruitment in ICUs: application to ventilator-induced diaphragm dysfunction

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: 72% positive predictive value

• Primary Outcome: Positive predictive value of LLM prescreening for VIDD trial eligibility

• Nnt Or Sensitivity: F1-score 0.82 for criterion-level performance; sensitivity and specificity not reported

• Confidence Interval: 95% CI 65-78%

Clinical Application

Requires significant IT infrastructure and ongoing expert validation; 86% reduction in review time suggests high efficiency gains Relevant for Australian ICUs conducting clinical trials; would need validation with Australian EHR systems and TGA trial requirements; cost-effectiveness analysis needed for PBS consideration of any resulting interventions ICU patients requiring mechanical ventilation ≥3 days in academic medical centres with structured discharge summaries

Abstract

BACKGROUND: Ventilator-induced diaphragm dysfunction (VIDD) is a frequent and under-recognized consequence of prolonged mechanical ventilation in intensive-care unit (ICU) patients. Identifying eligible candidates for clinical trials targeting VIDD remains a major operational challenge. This study evaluates the use of large language models (LLMs) to automate patient prescreening from ICU discharge summaries and estimate recruitment capacity for a future phase 2 trial. METHODS: We developed an LLM-based prescreening pipeline to assess trial eligibility criteria from ICU discharge summaries, which was deployed to screen all 2024 ICU stays. Stays that were flagged as potentially eligible underwent expert adjudication. An enriched set of 50 ICU stays was independently annotated by six clinicians to define a reference standard, which was used to evaluate criterion-level model performances using F1-scores. RESULTS: The best-performing model was GPT-OSS:120B with a criterion-level F1-score of 0.82. When applied to consecutive 1,342 ICU stays from Montpellier University Hospital in 2024, the selected model identified 532 patients with ≥ 3 days of mechanical ventilation. After applying exclusion criteria, 185 patients remained potentially eligible. Expert review confirmed 133 patients as eligible, resulting in a positive predictive value of 72% (95% CI 65-78). The LLM-assisted workflow resulted in an estimated 86% reduction in clinician review time. The LLM-based prescreening pipelines achieved criterion-level F1 scores ranging from 0.73 to 0.82, with GPT-OSS:120B demonstrating the highest performance. CONCLUSIONS: LLM-based prescreening offers a promising approach for identifying trial candidates in critical care, prioritizing candidates for clinician review. Future deployments should include targeted expert validation and ongoing monitoring to ensure safety and generalizability.

References

1. 1.Korvin, Kieffer, Loi, Z., Morquin, D., Jung, B., Pensier, J., Aarab, Y., Benchabane, N., Bendiab, E., Colombani, S., Lacampagne, A., Yauy, K., & Matecki, S. (2026). Large language models for optimizing clinical trial recruitment in ICUs: application to ventilator-induced diaphragm dysfunction. Critical Care, Article 12752. https://doi.org/10.1613/jair.1.12752