SAM2HIPT: a hybrid deep learning framework integrating SAM2 and HIPT with joint loss optimization for immunohistochemical cell nucleus segmentation

Clinical Snapshot

Key Findings

• P Value: Not reported

• Effect Size: Dice coefficients of 0.92 (BCData) and 0.91 (DeepLIIF)

• Primary Outcome: Nucleus segmentation accuracy measured by Dice coefficient

• Nnt Or Sensitivity: HD95 boundary errors of 1.05 and 1.10 pixels respectively

• Confidence Interval: Not reported

Clinical Application

Requires significant computational resources and technical expertise for implementation Could enhance digital pathology capabilities in Australian laboratories, potentially supporting TGA-approved diagnostic workflows and improving consistency in cancer diagnosis across regional centres Pathology laboratories performing immunohistochemistry analysis for cancer diagnosis and treatment assessment

Abstract

Nucleus segmentation in immunohistochemistry images plays a critical role in cancer diagnosis and treatment assessment. However, existing methods remain limited in segmentation accuracy and boundary delineation due to staining heterogeneity, densely packed cell distributions, and complex background interference. To address these challenges, this paper proposes a two-stage nucleus segmentation framework, termed SAM2HIPT. In the first stage, the pre-trained Segment Anything Model 2 (SAM2) is employed to generate initial segmentation predictions for input images, wherein the image encoder is kept frozen to preserve the pre-trained visual representation capacity while the mask decoder is fine-tuned to adapt to the characteristics of the pathological image domain; local texture, morphological, and boundary information are extracted through visual feature encoding to produce initial nucleus segmentation masks and spatial prior representations. In the second stage, the Hierarchical Image Pyramid Transformer is introduced to refine the initial segmentation results, performing multi-scale, multi-level feature representation and fusion of morphological, textural, and spatial structural information through a hierarchical vision Transformer architecture, thereby enhancing nuclear structural representation and boundary consistency. To enable collaborative optimization across both stages, a joint loss function is designed to impose unified constraints on segmentation accuracy and feature representation. Evaluated on two public histopathological benchmark datasets, BCData and DeepLIIF, the proposed method achieves Dice coefficients of 0.92 and 0.91, respectively, and HD95 boundary error values of 1.05 pixels and 1.10 pixels, demonstrating superior segmentation performance and robustness over multiple state-of-the-art baseline methods.

References

1. 1.Yao, Y., Li, S., Hu, Y., Xue, Y., Huang, J., He, J., & Li, S. (2026). SAM2HIPT: a hybrid deep learning framework integrating SAM2 and HIPT with joint loss optimization for immunohistochemical cell nucleus segmentation. Biomedical Physics & Engineering Express. https://doi.org/10.1088/2057-1976/ae63d6