A Survey on Interpretability in Visual Recognition

Clinical Snapshot

Key Findings

• P Value: Not applicable — no hypothesis testing performed

• Effect Size: Not applicable — no pooled effect size reported

• Primary Outcome: Taxonomic classification of XAI methods for visual recognition across four dimensions: intent, object, presentation, and methodology. No primary clinical outcome is defined.

• Nnt Or Sensitivity: Not applicable — no NNT, sensitivity, specificity, or hazard ratio reported. Quantitative benchmarks are technical XAI performance metrics (e.g., faithfulness, localisation accuracy) without clinical interpretability

• Confidence Interval: Not applicable — no confidence intervals reported

Clinical Application

The survey's findings are not directly implementable in clinical practice. They may inform procurement decisions for AI diagnostic systems, guide requirements for XAI features in clinical AI tools, or support regulatory submissions requiring model explainability documentation. Practical implementation requires translation by clinical informatics specialists. In Australia, the Therapeutic Goods Administration (TGA) regulates AI-based medical devices under the Software as a Medical Device (SaMD) framework, and the Australian Digital Health Agency (ADHA) has published guidance on responsible AI in health. This survey's taxonomy of XAI methods is conceptually relevant to TGA conformity assessment requirements for explainability and transparency in AI diagnostic tools. The RACGP has not issued specific guidelines on XAI requirements for clinical AI, but the Medical Board of Australia's guidance on AI emphasises clinician understanding of AI outputs — a domain where XAI methods surveyed here are directly relevant. PBS and TGA approval pathways for AI-assisted diagnostics (e.g., AI retinal screening, AI-assisted pathology) would benefit from the evaluation frameworks described, though direct clinical practice change cannot be recommended based on this survey alone. Not directly applicable to a defined patient population. Potentially relevant to clinical informaticists, radiologists, pathologists, and clinical AI governance teams seeking to understand the landscape of explainability methods for AI-assisted diagnostic tools

Abstract

Visual recognition models have achieved unprecedented success in various tasks. While researchers aim to understand the underlying mechanisms of these models, the growing demand for deployment in safety-critical areas like autonomous driving and medical diagnostics has accelerated the development of eXplainable AI (XAI). Distinct from generic XAI, visual recognition XAI is positioned at the intersection of vision and language, which represent the two most fundamental human modalities and form the cornerstones of multimodal intelligence. This paper provides a systematic survey of XAI in visual recognition by establishing a multi-dimensional taxonomy from a human-centered perspective based on intent, object, presentation, and methodology. Beyond categorization, we summarize critical evaluation desiderata and metrics, conducting an extensive qualitative assessment across different categories and demonstrating quantitative benchmarks within specific dimensions. Furthermore, we explore the interpretability of Multimodal Large Language Models and practical applications, identifying emerging trends and opportunities. By synthesizing these diverse perspectives, this survey provides an insightful roadmap to inspire future research on the interpretability of visual recognition models.

References

1. 1.Wan, Q., Gao, C., Wang, R., & Chen, X. (2026). A survey on interpretability in visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2026.3672629