EviVLM: When Evidential Learning Meets Vision Language Model for Medical Image Segmentation

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Not specified in abstract

• Primary Outcome: State-of-the-art segmentation performance on three public medical image datasets

• Nnt Or Sensitivity: Segmentation accuracy metrics not specified

• Confidence Interval: Not reported

Clinical Application

Limited by computational requirements and need for technical expertise in implementation Potential application in Australian radiology departments, but would require TGA software validation and integration with existing PACS systems. Cost-effectiveness analysis needed for PBS consideration if used in diagnostic workflows. Medical imaging departments requiring automated segmentation across multiple modalities

Abstract

The disparity between image and text representations, often referred to as the modality gap, remains a significant obstacle for Vision Language Models (VLMs) in medical image segmentation. This gap complicates multi-modal fusion, thereby restricting segmentation performance. To address this challenge, we propose Evidence-driven Vision Language Model (EviVLM)-a novel paradigm that integrates Evidential Learning (EL) into VLMs to systematically measure and mitigate the modality gap for enhanced multi-modal fusion. To drive this paradigm, an Evidence Affinity Map Generator (EAMG) is proposed to collect complementary cross-modal evidences by learning a global cross-modal affinity map, thus refining modality-specific evidence embedding. An Evidence Differential Similarity Learning (EDSL) is further proposed to collect consistent cross-modal evidences by performing Bias-Variance Decomposition on differential matrix derived from bidirectional similarity matrices between image and text evidence embeddings. Finally, the subjective logic is used for mapping the collected evidences to opinions, and the Dempster-Shafer's theory based combination rule is introduced for opinion aggregation, thereby quantifying the modality gap and facilitating effective multi-modal integration. Experimental results on three public medical image segmentation datasets validate that the proposed EviVLM can achieve state-of-the-art performance. Code is available at: https://github.com/QingtaoPan/EviVLM.

References

1. 1.Pan, Q., Li, Z., Yang, G., Yang, Q., & Ji, B. (2026). EviVLM: When Evidential Learning Meets Vision Language Model for Medical Image Segmentation. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/TMI.2025.3622492