External Validation, Recalibration, and Extension of a Prediction Model of Early Acute Kidney Injury in Critically Ill Children Using Multicenter Data

Clinical Snapshot

Key Findings

• P Value: Not reported for primary discrimination measures

• Effect Size: AUROC improved from 0.65-0.78 (recalibrated model) to 0.80-0.88 (extended model)

• Primary Outcome: Early acute kidney injury within 72 hours of PICU admission

• Nnt Or Sensitivity: AUPRC improved from 0.10-0.12 to 0.13-0.22, indicating better precision-recall balance

• Confidence Interval: Not explicitly reported for discrimination metrics

Clinical Application

High feasibility as model uses routinely collected EHR data available within 12 hours of admission Highly relevant to Australian PICUs; could integrate with existing clinical information systems. No specific regulatory barriers as uses standard clinical parameters. Aligns with ANZPIC quality improvement initiatives Critically ill children admitted to pediatric intensive care units, particularly applicable to tertiary pediatric hospitals

Abstract

BACKGROUND: Acute kidney injury (AKI) is common in critically ill children and is associated with high morbidity and mortality. Risk prediction models designed for clinical decision support implementation can facilitate early identification and proactive mitigation of AKI risk. Existing models have primarily been validated using single-center data, partly because of the lack of appropriately detailed multicenter datasets. OBJECTIVE: To determine the performance of a single-center model to predict new AKI at 72 hours of ICU admission in children across two multicenter datasets and refine this model to improve prediction performance while maintaining acceptable alert burden. DERIVATION AND VALIDATION COHORTS: We analyzed two datasets: the Pediatric Learning Health System Network-Virtual Pediatric Systems (PEDSNET-VPS) dataset, created through the linkage of PEDSnet electronic health record (EHR) extraction with VPS (LLC, http://www.myvps.org), and the PICU Data Collaborative dataset, created through EHR extraction and harmonization from eight participating institutions. We divided each dataset into a derivation and test split. PREDICTION MODEL: We first recalibrated an existing single-center model and measured discrimination (area under the receiver operating characteristic curve [AUROC] and area under the precision-recall curve [AUPRC]) and performance at multiple cutpoints. We next added features available at 12 hours of ICU admission, optimizing by precision and recall. We measured discrimination and performance at multiple cutpoints and identified the features contributing most to the risk score. RESULTS: In total we analyzed 186,540 ICU admissions. We found early AKI by serum creatinine criteria within 72 hours of admission in 2.2-2.7%. Initial recalibration of an existing single-center model demonstrated poor discrimination (AUROC 0.65-0.78; AUPRC 0.10-0.12). Following the addition of new features, the model had higher AUROC (0.80-0.88) and AUPRC (0.13-0.22). CONCLUSIONS: In this first use of two new multicenter datasets, we found improved performance in a model designed using features available at 12 hours of ICU admission, balancing sensitivity and precision to predict patients at risk for AKI development.

References

1. 1.Dziorny, A. C., Drury, S., Clark, A., Farris, R. W. D., Nishisaki, A., Cornell, T. T., Tawfik, D. S., Bennett, T. D., Shah, S. S., Weiss, S. L., Mohamed, T., Shah, N., McMahon, J., Muthu, N., Wetzel, R. C., Zand, M., & Sanchez-Pinto, L. N. (2026). External validation, recalibration, and extension of a prediction model of early acute kidney injury in critically ill children using multicenter data. Critical Care Explorations. https://doi.org/10.1097/CCE.0000000000001425