A multi-task cross-attention strategy to segment and classify polyps
Clinical Snapshot
PICO Framework
| P — Population | Colonoscopy images from BKAI-IGH dataset (1200 samples: 1000 white-light imaging, 200 NICE samples) with fine-grained segmentation masks |
| I — Intervention | Multi-task deep learning model using multi-head cross-attention for polyp segmentation and malignancy stratification |
| C — Comparator | Not clearly specified - appears to compare against existing state-of-the-art methods on public datasets |
| O — Outcomes | Primary: Intersection over Union (IoU) for segmentation, recall for detection. Secondary: Malignancy stratification accuracy, generalization on external datasets |
Bottom Line
This study presents a promising computer vision approach for automated polyp detection and malignancy assessment during colonoscopy. The multi-task model achieved good performance metrics (83.5% IoU, 94% recall) on a validated dataset and demonstrated generalization capability. However, the clinical utility remains uncertain due to limited real-world validation and lack of comparison with expert gastroenterologist performance. While the technology shows potential to assist in polyp characterization and reduce missed lesions, significant implementation challenges remain including regulatory approval, system integration, and validation in diverse clinical settings. Australian gastroenterologists should await further clinical validation studies before considering adoption, particularly given the critical nature of accurate polyp assessment in colorectal cancer prevention programs.
Key Findings
P Value: Not reported
Effect Size: 83.5% average IoU for segmentation
Primary Outcome: Polyp segmentation accuracy measured by Intersection over Union (IoU)
Nnt Or Sensitivity: 94% recall for polyp detection
Confidence Interval: Not reported
Clinical Application
Requires integration with existing colonoscopy systems and validation in real-time clinical settings before implementation Could support Australia's National Bowel Cancer Screening Program and align with RACGP guidelines for colorectal cancer screening, but would require TGA approval and integration with local endoscopy systems Patients undergoing colonoscopy screening or surveillance, particularly those at risk for colorectal cancer
Abstract
Polyps are the main biomarkers for diagnosing colorectal cancer. Their early detection and accurate characterization during colonoscopy procedures rely on expert observations. Nevertheless, such a task is prone to errors, particularly in morphological characterization. This work proposes a multi-task representation capable of segmenting polyps and stratifying their malignancy from individual colonoscopy frames. The approach employs a deep representation based on multi-head cross-attention, refined with morphological characterization learned from independent maps according to the degree of polyp malignancy. The proposed method was validated on the BKAI-IGH dataset, comprising 1200 samples (1000 white-light imaging and 200 NICE samples) with fine-grained segmentation masks. The results show an average IoU of 83.5% and a recall of 94%. Additionally, external dataset validation demonstrated the model's generalization capability. Inspired by conventional expert characterization, the proposed method integrates textural and morphological observations, allowing both tasks, polyp segmentation and the corresponding malignancy stratification. The proposed strategy achieves the state-of-the-art performance in public datasets, showing promising results and demonstrating its ability to generate a polyp representation suitable for multiple tasks.
References
- 1.Sierra-Jerez, F., Ruiz, L., & Martínez, F. (2026). A multi-task cross-attention strategy to segment and classify polyps. Biomedical Physics & Engineering Express. https://doi.org/10.1088/2057-1976/ae2b78
Related Research
The American journal of pathology
An Integrated Machine Learning and Genomic Framework for Precise Detection of Gastric Cancer
3 Aug 2026
Clinical gastroenterology and hepatology : the official clinical practice journal of the American Gastroenterological Association
Artificial Intelligence Tools for Gastrointestinal Research: A Practical Guide
2 Aug 2026
Current opinion in critical care
Smart feeding: the role of artificial intelligence and integrated nutrition platforms in the ICU
2 Aug 2026
This content is for educational purposes for healthcare professionals only and does not constitute clinical advice. Clinical decisions should be based on individual patient assessment, current guidelines, and appropriate specialist consultation. Editorial Standards · Privacy Policy · Terms of Service