An approximation theory perspective on machine learning
Clinical Snapshot
PICO Framework
| P — Population | Not applicable in the clinical sense; the 'population' is the domain of machine learning models and their theoretical underpinnings, including neural networks and kernel-based methods applied to datasets drawn from unknown probability distributions |
| I — Intervention | Approximation theory frameworks applied to machine learning, including shallow/deep networks, manifold-based approximation, physics-informed neural surrogates, neural operators, and transformer architectures |
| C — Comparator | Current machine learning practice and theoretical foundations that do not centrally incorporate approximation theory |
| O — Outcomes | Generalisation capability of trained models to unseen data; theoretical guarantees for function approximation; classification accuracy via signal separation frameworks; open research problems identified |
Bottom Line
This is a theoretical mathematical review, not a clinical study, and should be evaluated accordingly. The paper makes a coherent and intellectually valuable argument that approximation theory — the mathematical study of how well functions can be represented by simpler models — has been insufficiently integrated into the theoretical foundations of modern machine learning. The authors identify a consequential practical problem: without rigorous approximation-theoretic grounding, it remains unclear how well trained machine learning models will generalise to new data. They propose two potentially significant theoretical advances: a framework for function approximation on unknown manifolds that avoids computationally expensive eigen-decomposition, and a reframing of classification problems as signal separation tasks. For senior clinicians and health informaticists, the key takeaway is cautionary rather than prescriptive. The paper reinforces that the theoretical guarantees underpinning many clinical AI tools remain incomplete, and that generalisation to unseen patient populations cannot be assumed. This is not a paper that changes clinical practice today, but it contributes to the intellectual scaffolding needed to build more trustworthy clinical AI in the future. Evidence rating: Weak for direct clinical application; the work is theoretically sound but empirically unvalidated.
Key Findings
P Value: Not applicable — no hypothesis testing performed
Effect Size: Not applicable — no empirical effect sizes reported; the paper presents theoretical approximation error bounds and mathematical propositions
Primary Outcome: Theoretical characterisation of the disconnect between approximation theory and machine learning practice, with proposals for bridging this gap through manifold-based approximation and signal separation frameworks for classification
Nnt Or Sensitivity: Not applicable — no clinical diagnostic or therapeutic outcomes; the analogous theoretical metric would be approximation error rates and convergence guarantees, which are discussed qualitatively but not benchmarked empirically
Confidence Interval: Not applicable — no confidence intervals; mathematical proofs provide deterministic or probabilistic bounds rather than frequentist intervals
Clinical Application
The theoretical frameworks proposed (manifold-based approximation without Laplace-Beltrami eigen-decomposition; classification as signal separation) are mathematically described but not yet implemented or validated in clinical machine learning pipelines. Feasibility for clinical deployment requires substantial further empirical work, software implementation, and prospective validation in healthcare datasets. This paper has no direct implications for current Australian clinical practice, PBS listings, or TGA-regulated medical devices. However, it is indirectly relevant to the Australian Digital Health Agency's AI strategy and the TGA's evolving regulatory framework for Software as a Medical Device (SaMD). The theoretical insights regarding generalisation failure in machine learning models are pertinent to Australian clinicians and health informaticists evaluating AI-based clinical decision support tools. The RACGP's position on AI in general practice emphasises the need for transparent, validated, and generalisable models — concerns directly addressed (though not resolved) by this paper. Australian researchers in health AI at institutions such as the Australian Institute of Machine Learning (AIML) or the Doherty Institute's computational biology groups may find the manifold approximation frameworks relevant to high-dimensional clinical data problems. Not directly applicable to any specific patient population. Indirectly relevant to developers and researchers building machine learning models for clinical applications, including medical imaging, clinical decision support, genomics, and electronic health record analysis. The theoretical frameworks discussed could inform the design of more generalisable clinical AI tools.
Abstract
A central problem in machine learning is often formulated as follows: Given a dataset [Formula: see text] , which is a sample drawn from an unknown probability distribution, the goal is to construct a functional model f such that f(x) ≈ y for any (x, y) drawn from the same distribution. Neural networks and kernel-based methods are commonly employed for this task due to their capacity for fast and parallel computation. The approximation capabilities, or expressive power, of these methods have been extensively studied over the past 35 years. In this paper, we will present examples of key ideas in this area found in the literature. We will discuss emerging trends in machine learning including the role of shallow/deep networks, approximation on manifolds, physics-informed neural surrogates, neural operators, and transformer architectures. Despite function approximation being a fundamental problem in machine learning, approximation theory does not play a central role in the theoretical foundations of the field. One unfortunate consequence of this disconnect is that it is often unclear how well trained models will generalize to unseen or unlabeled data. In this review, we examine some of the shortcomings of the current machine learning framework and explore the reasons for the gap between approximation theory and machine learning practice. We will then review some of recent work that achieves function approximation on unknown manifolds without the need to learn specific manifold features, such as the eigen-decomposition of the Laplace-Beltrami operator or atlas construction. In many machine learning problems, particularly classification tasks, the labels yj are drawn from a finite set of values. We summarize another recent paper that establishes a deep connection between signal separation problems and classification problems, proposing that classification tasks should be approached as instances of signal separation. We conclude by identifying several open research problems that warrant further investigation.
References
- 1.Mhaskar, H. N., Tsoukanis, E., & Jagtap, A. D. (2026). An approximation theory perspective on machine learning. Neural Networks: The Official Journal of the International Neural Network Society. https://doi.org/10.1016/j.neunet.2026.108841
Related Research
Journal of medical Internet research
Advancing Human-Centered AI in Clinical Decision Support: Sociocognitive Human-in-the-Loop Study in HIV Care
2 Aug 2026
Neurological research
Federated deep learning model for epilepsy seizure detection using electroencephalogram (EEG) signal
2 Aug 2026
Neural networks : the official journal of the International Neural Network Society
MFBD: Model-free backdoor defense based on vision-language pre-trained models
2 Aug 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service