Reproducible workflow for online artificial intelligence in digital health

Clinical Snapshot

Key Findings

• Effect Size: Not applicable - methodological framework

• Primary Outcome: Proposed reproducible scientific workflow for online AI in digital health

• Nnt Or Sensitivity: Not applicable - workflow development study

• Confidence Interval: Not provided

Clinical Application

Depends on institutional resources and technical infrastructure for implementation Relevant to Australian Digital Health Agency initiatives, TGA software as medical device regulations, and NHMRC research reproducibility guidelines Digital health researchers, AI developers, clinical informaticians implementing online learning algorithms

Abstract

Online artificial intelligence (AI) algorithms are an important component of digital health interventions. These online algorithms are designed to continually learn and improve their performance as streaming data are collected on individuals. Deploying online AI presents a key challenge: balancing adaptability of online AI with reproducibility. Online AI in digital interventions is a rapidly evolving area, driven by advances in algorithms, sensors, software and devices. Digital health intervention development and deployment is a continuous process, where implementation-including the AI decision-making algorithm-is interspersed with cycles of re-development and optimization. Each deployment informs the next, making iterative deployment a defining characteristic of this field. This iterative nature underscores the importance of reproducibility: data collected across deployments must be accurately stored to have scientific utility, algorithm behaviour must be auditable and results must be comparable over time to facilitate scientific discovery and trustworthy refinement. This article proposes a reproducible scientific workflow for developing, deploying and analysing online AI decision-making algorithms in digital health interventions. Grounded in practical experience from multiple real-world deployments, this workflow addresses key challenges to reproducibility across all phases of the online AI algorithm development life cycle. This article is part of the theme issue 'Statistical workflow'.