Machine Learning-Based Prediction Model for 30-Day Emergency Department Revisits in a Medically Underserved Tertiary Hospital: Formative Retrospective Cohort Study

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: AUC 0.90 (95% CI 0.88-0.92) for XGBoost model

• Primary Outcome: 30-day emergency department revisit

• Nnt Or Sensitivity: Sensitivity 0.94, specificity 0.69, 25-fold difference in revisit rates between risk groups

• Confidence Interval: 95% CI 0.88-0.92

Clinical Application

Requires electronic health record integration and ML infrastructure Relevant for Australian EDs, particularly in regional/rural settings. Would need validation in Australian healthcare context and integration with existing ED information systems Adult emergency department patients in tertiary hospital settings

Abstract

BACKGROUND: Emergency department (ED) revisits are critical quality indicators, particularly in medically underserved areas, where traditional prediction tools show limited performance. Machine learning (ML) approaches may offer improved predictive performance for identifying high-risk patients. OBJECTIVE: This formative study aimed to develop and validate an ML-based model for predicting 30-day ED revisits using electronic health records from a tertiary hospital serving a medically underserved area in South Korea and to evaluate its clinical utility through interpretability analysis and risk stratification. METHODS: This retrospective cohort study analyzed 36,230 adult patients visiting the Gangneung Asan Hospital ED in 2023. We developed and compared 3 ML models (extreme gradient boosting [XGBoost], random forest, and ElasticNet) using electronic health records. Model interpretability was ensured through Shapley additive explanations (SHAP) analysis, and clinical utility was evaluated through 5-tier risk stratification. RESULTS: Among 36,230 patients, 798 (2.2%) revisited within 30 days. XGBoost achieved superior performance with an area under the receiver operating characteristic curve of 0.90 (95% CI 0.88-0.92), a sensitivity of 0.94, and a specificity of 0.69. The SHAP analysis identified ED length of stay, oxygen saturation, systolic blood pressure, computed tomography performance, antibiotic use, and liver disease as key predictors. Risk stratification demonstrated a 25-fold difference in the actual revisit rates between the lowest (152/8450, 1.8%) and the highest (686/1500, 45.7%) risk groups. CONCLUSIONS: The XGBoost model demonstrated excellent predictive performance with high interpretability for 30-day ED revisit predictions. The implementation of this model could enable risk-stratified interventions and more efficient resource allocation in medically underserved settings, potentially reducing unnecessary revisits and improving patient outcomes. This formative study establishes feasibility and provides a foundation for future multicenter validation studies in similar medically underserved settings.