Evaluation of a deep learning model applied to chest X-rays of patients with suspected pneumonia presenting to the emergency department designed to predict admission risk: a retrospective training and prospective non-intervention validation.

Clinical Snapshot

Key Findings

• P Value: p<0.001 for difference in scores between admitted vs discharged patients

• Effect Size: AUC 0.795 (overall), 0.828 (pneumonia-specific complaints), 0.755 (non-pneumonia complaints)

• Primary Outcome: Inpatient admission prediction

• Nnt Or Sensitivity: Accuracy 0.751 (PSI-anchored), 0.729 (CRB-anchored); Specificity 0.803 and 0.968 respectively

• Confidence Interval: Not reported

Clinical Application

Requires integration with radiology systems and real-time processing capabilities Could support ED decision-making in Australian hospitals, particularly relevant given ED overcrowding issues. Would need TGA approval as medical device software. Integration with existing PACS systems feasible in major centres. Adult ED patients with suspected pneumonia requiring chest X-ray

Abstract

OBJECTIVES: To develop, train and test a deep learning model Image-based PROgnostication applied to Chest X Rays (IPRO-X) tool that predicts the inpatient (IP) admission risk in patients with suspected pneumonia presenting to the emergency department (ED). DESIGN: The study consists of a retrospective (training) and prospective non-interventional shadow deployment (validation) of a deep learning model. SETTING: Three-hospital tertiary care system with two EDs. PARTICIPANTS: Consecutive adult patients (18 years old or older) who presented to the ED from December 2022 to February 2023 with a chief clinical complaint potentially related to pneumonia were reviewed for eligibility (n=5567), with a final number of 3677 included in the validation study. PRIMARY AND SECONDARY OUTCOME MEASURES: IPRO-X was developed using standard two-dimensional convolutional neural network InceptionNet architecture and processes chest radiographs (CXRs) acquired at admission generating a continuous value from 0 (IP-negative) to 1 (IP-positive). We examined IPRO-X's ability to predict IP admission using accuracy, area under the curve (AUC), receiver operating characteristic, sensitivity, specificity, positive predictive value and negative predictive value. IPRO-X scores were compared against observed outcomes (admission and discharge) to determine clinical utility in an ED setting. Four thresholds were defined from the retrospective phase: Youden optimal, highest specificity and thresholds equivalent to 30-day mortality rates of Pneumonia Severity Index (PSI) Risk class IV and Confusion, Respiratory Rate, Blood Pressure and Age 65 or older Score (CRB-65) scores 1 and 2. Performance was also analysed per group of chief complaints. RESULTS: In the validation set, 3677 patients (1777 (48%) female, median age 56 years (min 18, max 99)) were included. IPRO-X predicted IP admissions with an AUC of 0.795, and in the pneumonia-specific chief complaints, AUC increased to 0.828, whereas in non-related pneumonia chief complaints, the AUC decreased to 0.755. IPRO-X score was significantly higher in admitted patients compared with discharged patients (p<0.001). The accuracy of IPRO-X when using the PSI-anchored threshold was 0.751 and with the CRB-anchored threshold was 0.729, with a specificity of 0.803 and 0.968, respectively. CONCLUSIONS: IPRO-X applied to CXR of patients with signs and symptoms commonly related to pneumonia in the ED can accurately predict IP admission.