Comparing machine and deep learning models for pediatric anxiety classification using structured EHRs and area-based measures of health data

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Best performing model (Dipole): AUROC 0.857 (EHR+ABMH), 0.853 (EHR only) for 8-year-olds

• Primary Outcome: AUROC for anxiety disorder classification

• Nnt Or Sensitivity: XGBoost baseline: AUROC 0.817 (EHR), 0.816 (EHR+ABMH)

• Confidence Interval: Not reported

Clinical Application

Requires sophisticated ML/DL infrastructure and validated EHR data quality - challenging for routine clinical implementation Limited applicability without validation in Australian EHR systems; would require TGA approval for clinical decision support tools and integration with existing pediatric mental health pathways Pediatric patients in healthcare systems with comprehensive structured EHR data

Abstract

OBJECTIVE: This retrospective, case-control study with internal validation evaluates the performance of machine learning (ML) and deep learning (DL) models in classifying pediatric patients at risk for anxiety disorders using structured electronic health records (EHRs) and area-based measures of health (ABMH). The aim is to enable proactive care by monitoring potential anxiety onset across developmental stages. METHODS: We trained a series of ML models (Logistic Regression, Decision Tree, Random Forest, K-Nearest Neighbors, XGBoost) and DL models (LSTM, GRU, RETAIN, Dipole) using structured EHR data from 30-day windows prior to diagnosis. Two datasets were used per age group: one with structured EHR data only, and another including both EHR and ABMH data. ML models were trained using short-term cross-sectional features, while DL models leveraged full longitudinal patient histories. Performance was assessed using AUROC, AUPRC, PPV, NPV, F1 score, and accuracy. Due to differences in input scope, model performance reflects both algorithmic and temporal design differences and is not intended as a direct comparison between ML and DL. RESULTS: ML models offered strong baseline performance, with XGBoost achieving AUROC scores of 0.817 (EHR) and 0.816 (EHR+ABMH) for 8-year-olds. Adding ABMH features did not significantly improve performance. DL models, particularly RETAIN and Dipole, achieved the highest AUROC values (e.g., Dipole: 0.853 with EHR, 0.857 with EHR+ABMH for 8-year-olds), outperforming other DL and ML models within their respective design constraints. CONCLUSION: Both ML and DL models successfully identified likely anxiety onset using structured EHR data. DL models using longitudinal data achieved the highest performance, while XGBoost provided a robust ML baseline. The minimal impact of ABMH features highlights integration challenges, and performance variation across ages emphasizes the need for age-stratified modeling approaches.

References

1. 1.Lee, E. W., Choo, S., Maguire, D., Shivanna, A., Santel, D., Bhatnagar, S., Goethert, I., Patterson, K., Gholap, J., Hanson, H. A., Chandrashekar, M., Ammerman, R. T., Pestian, J. P., Glauser, T., Brokamp, C., Strawn, J. R., Kapadia, A. J., & Agasthya, G. (2026). Comparing machine and deep learning models for pediatric anxiety classification using structured EHRs and area-based measures of health data. PLoS One. https://doi.org/10.1371/journal.pone.0324673