Empirical evaluation of variability and multi-institutional generalizability of deep learning survival models: application to renal cancer CT scans

Clinical Snapshot

Key Findings

• P Value: Not reported in abstract

• Effect Size: C-index 0.74 with covariate-balanced partitioning vs 0.56 with random partitioning

• Primary Outcome: Overall survival prediction using CT-based deep learning models

• Nnt Or Sensitivity: Hazard ratio 5.08 (optimal model) vs 0.29 (random partitioning); 4.76% improvement in c-index with augmentation

• Confidence Interval: Not reported in abstract

Clinical Application

Requires sophisticated computational infrastructure and technical expertise; integration into routine practice would need significant implementation planning Relevant to Australian oncology practice where CT imaging is standard for renal cancer staging and monitoring. Would need validation in Australian cohorts and integration with existing cancer care pathways Patients with renal cancer undergoing CT imaging for prognostic assessment

Abstract

Deep learning survival (DLS) modeling has shown significant promise across multiple disease conditions for predicting patient outcomes via biomedical data, but remain relatively under-explored in conjunction with medical imaging. This poses the question of what methodological choices promote model generalization, especially when predicting continuous survival outcomes in external validation cohorts. In this work, we systematically examined the impact of data partitioning, data order, model initialization, and augmentation strategies on the generalization of CT-based DLS survival models in renal cancers. A multi-institutional CT cohort was assembled, leveraging public repositories such as TCGA-KIRC, TCGA-KICH, TCGA-KIRP, and KITS19; totaling 525 patients from across 9 institutions. A 3D ResNet-18 model was trained to predict overall patient survival with a Cox loss function and evaluated across three controlled experiments: (i) random versus covariate-balanced data partitioning, (ii) randomizing vs fixing data ordering and initial model weights, and (iii) varying intensity, spatial, and noise based augmentations; evaluated in terms of concordance index (c-index) and hazard ratio (HR). Intelligent covariate-balanced data partitioning demonstrated optimal performance on an external validation set (c-index 0.74, HR 5.08) compared to random partitioning (c-index 0.56, HR 0.29). Different model initializations significantly impacted the average performance and variance across replicate trainings, while data ordering had little to no effect. Combining multiple augmentations produced the best performance on external validation (c-index +4.76%, HR +44.39%) compared to no augmentations, while most impactful individual augmentation type was additive gaussian noise (c-index +3.17%, HR +23.47%). Our results demonstrate that careful selection of strategies for data partitioning and ordering, weight initialization, and image augmentation are critical for developing robust and generalizable DL models for continuous survival prediction using medical imaging data.

References

1. 1.Flannery, B., Brennan, T., DeSilvio, M., Thomas, H., Hariri, M., Sadri, A. R., Heller, N., Weight, C., & Viswanath, S. E. (2026). Empirical evaluation of variability and multi-institutional generalizability of deep learning survival models: application to renal cancer CT scans. Computers in Biology and Medicine. https://doi.org/10.1007/s10462-020-09825-6