Discovery of an Artificial Intelligence Label Feedback Loop: How the Success of a Clinically Implemented Artificial Intelligence Algorithm Has Created an Unforeseen Challenge to Algorithm Retraining

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: 10% reduction in concordance rate

• Primary Outcome: V2 model showed 10% lower concordance than V1 on recent production data

• Nnt Or Sensitivity: V2 outperformed V1 specifically for less-common stone types when evaluated on human-only labels

• Confidence Interval: Not reported

Clinical Application

Highly feasible - provides framework for identifying and mitigating AI feedback loops Relevant to TGA-approved AI medical devices and pathology laboratories under NPAAC accreditation; important for ongoing AI validation requirements Clinical laboratories implementing AI-augmented diagnostic tests

Abstract

BACKGROUND: Artificial intelligence (AI) augmented laboratory tests can improve quality, efficiency, cost-effectiveness, and staff satisfaction. However, the clinical success of these tests can introduce unforeseen challenges for model retraining. This study describes the discovery of an "AI label feedback loop" in a clinically implemented AI-augmented kidney stone composition test. METHODS: An AI-augmented kidney stone composition test (V1) has been previously deployed for clinical kidney stone characterization. After several years of clinical use, a retrained model (V2) was developed using 6 times more data. Model performance of both V1 and V2 were evaluated across 3 datasets: a recent production validation (hold-out) set (mostly AI-influenced labels), the original V1 validation set (pre-AI, entirely human-labeled), and a subset of recent cases with exclusively human-generated or human-corrected labels. RESULTS: V2 demonstrated a 10% lower concordance rate than V1 when evaluated on the recent production hold-out set, despite a much larger training dataset. Performance between V1 and V2 was similar when applied to the pre-AI validation set. Notably, V2 outperformed V1 on the recent subset of cases with human-only or human-corrected labels, particularly for less-common stone types. These findings revealed an AI label feedback loop, confounding retraining and evaluation. CONCLUSIONS: The integration of AI into clinical practice can potentially influence reported test results, complicating the development and evaluation of future models. To mitigate AI label feedback loops, ongoing human annotation and careful validation set construction are essential. These strategies can ensure reliable performance assessment and support the safe evolution of clinical AI systems.

References

1. 1.Day, P. L., Wieczorek, M. A., Rokke, D., Lantz, E., Bornhorst, J. A., Jannetto, P. J., & Carter, R. E. (2026). Discovery of an Artificial Intelligence Label Feedback Loop: How the Success of a Clinically Implemented Artificial Intelligence Algorithm Has Created an Unforeseen Challenge to Algorithm Retraining. Clinical Chemistry. https://doi.org/10.1093/clinchem/hvag001