Prediction of in-hospital cardiac arrest on general wards using calibrated machine learning

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: AUROC 0.89; Average Precision (AP) 0.88 in matched temporal test set (2021 cohort). Independent predictors: atrial fibrillation OR 6.30, heart failure OR 2.98, end-stage renal disease OR 2.54, potassium OR 1.81 per 1 mEq/L, WBC OR 1.06 per 1 k/µL, SpO₂ OR 1.30 per 1% decrease in final 8-hour window

• Primary Outcome: Prediction of in-hospital cardiac arrest (IHCA) on general wards using a calibrated XGBoost model with time-windowed clinical features

• Nnt Or Sensitivity: At sensitivity 0.95: specificity 0.47 in matched test set (50% prevalence). Derived LR+ ≈ 1.79; LR− ≈ 0.11. Note: PPV and NPV are not generalisable from matched-design metrics — real-world PPV at true IHCA prevalence (~0.3–0.5%) would be substantially lower, likely <2%, generating high false-alert burden

• Confidence Interval: 95% CI reported only for SpO₂ predictor: OR 1.30 (95% CI 1.08–1.54). No CIs reported for AUROC, AP, sensitivity, or specificity

Clinical Application

Deployment requires integration with hospital EHR systems capable of real-time extraction of vital signs, laboratory results, and comorbidity data across defined time windows. The three-window temporal feature engineering adds computational complexity compared to single-point early warning scores. Isotonic calibration requires periodic recalibration as patient population and clinical practices evolve. Alert fatigue at specificity 0.47 (in an already enriched cohort) would be a major implementation barrier in real-world settings with low IHCA base rates. Australian general wards currently use the National Early Warning Score (NEWS2) or hospital-specific track-and-trigger systems endorsed by the Australian Commission on Safety and Quality in Health Care (ACSQHC) under the Recognising and Responding to Clinical Deterioration Standard. This model is not TGA-listed as a medical device software (SaMD) and would require TGA regulatory assessment under the Software as a Medical Device framework before clinical deployment. The RACGP and ACSQHC have emphasised reducing alert fatigue in deterioration systems — the 0.47 specificity in a matched cohort raises serious concerns for Australian ward environments. Prospective multicentre validation in Australian hospitals with diverse case-mix (including rural and regional settings) would be required before any consideration of adoption. PBS relevance is not applicable to this diagnostic/monitoring tool. Adult general ward inpatients at risk of IHCA, particularly those with atrial fibrillation, heart failure, end-stage renal disease, electrolyte disturbances, or declining SpO₂. Patients with DNR orders were excluded from this study and the model is not validated for this group.

Abstract

OBJECTIVE: To identify time-windowed clinical predictors of in-hospital cardiac arrest (IHCA) and develop a temporally validated, calibrated machine-learning early warning for general wards. METHODS: Retrospective matched case-control study at a tertiary hospital in Taiwan (2019-2021) including 115 IHCA cases and 115 controls matched on age and ward. Patients with do-not-resuscitate (DNR) orders were excluded. Demographics, comorbidities and laboratory results were extracted from electronic health records. Vital signs and level of consciousness were sampled in 16-24, 8-16 and 1-8-hour windows before the index time and summarised as the modified early warning score (MEWS). Models were trained with fivefold cross-validation and isotonic probability calibration and tested on a temporally held-out 2021 cohort. RESULTS: Atrial fibrillation (OR 6.30), heart failure (OR 2.98), end-stage renal disease (OR 2.54), potassium (OR 1.81 per 1 mEq/L) and white blood count (OR 1.06 per 1 k/µL) were independent predictors. MEWS was associated with IHCA up to 16 hours, whereas in the final 8-hour lower SpO₂ predominated (OR 1.30 per 1% decrease; 95% CI 1.08 to 1.54). Calibrated XGBoost achieved Area Under the Receiver Operating Characteristic (AUROC) curve 0.89 and average precision (AP) 0.88; at sensitivity 0.95, specificity was 0.47 in the matched test set. CONCLUSIONS: Combining comorbidity burden, laboratory indices and time-windowed physiology enabled high-sensitivity IHCA warning. Because matching inflates event prevalence, AP, positive predictive value and calibration require recalibration to real-world prevalence; prospective multicentre validation is needed.