CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for
  Multi-Organ Segmentation
Multi-organ medical segmentation is a crucial component of medical image
processing, essential for doctors to make accurate diagnoses and develop
effective treatment plans. Despite significant progress in this field, current
multi-organ segmentation models often suffer from inaccurate details,
dependence on geometric prompts and loss of spatial information. Addressing
these challenges, we introduce a novel model named CRISP-SAM2 with CRoss-modal
Interaction and Semantic Prompting based on SAM2. This model represents a
promising approach to multi-organ medical segmentation guided by textual
descriptions of organs. Our method begins by converting visual and textual
inputs into cross-modal contextualized semantics using a progressive
cross-attention interaction mechanism. These semantics are then injected into
the image encoder to enhance the detailed understanding of visual information.
To eliminate reliance on geometric prompts, we use a semantic prompting
strategy, replacing the original prompt encoder to sharpen the perception of
challenging targets. In addition, a similarity-sorting self-updating strategy
for memory and a mask-refining process is applied to further adapt to medical
imaging and enhance localized details. Comparative experiments conducted on
seven public datasets indicate that CRISP-SAM2 outperforms existing models.
Extensive analysis also demonstrates the effectiveness of our method, thereby
confirming its superior performance, especially in addressing the limitations
mentioned earlier. Our code is available at:
https://github.com/YU-deep/CRISP\_SAM2.git.
6864dfe9d59a9eda59024ba1
Xinlei Yu
claimed_verified
2025-07-02T07:38:35.223Z
67d63e228d5c7a132cbcf39b
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ynwA3Sya5irwMRCmSeLiC.png
neil yu
user
yxl66666
6864dfe9d59a9eda59024ba2
Chanmiao Wang
6864dfe9d59a9eda59024ba3
Hui Jin
6864dfe9d59a9eda59024ba4
Ahmed Elazab
6864dfe9d59a9eda59024ba5
Gangyong Jia
6864dfe9d59a9eda59024ba6
Xiang Wan
6864dfe9d59a9eda59024ba7
Changqing Zou
6864dfe9d59a9eda59024ba8
Ruiquan Ge
CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for
  Multi-Organ Segmentation
Multi-organ medical segmentation is a crucial component of medical image
processing, essential for doctors to make accurate diagnoses and develop
effective treatment plans. Despite significant progress in this field, current
multi-organ segmentation models often suffer from inaccurate details,
dependence on geometric prompts and loss of spatial information. Addressing
these challenges, we introduce a novel model named CRISP-SAM2 with CRoss-modal
Interaction and Semantic Prompting based on SAM2. This model represents a
promising approach to multi-organ medical segmentation guided by textual
descriptions of organs. Our method begins by converting visual and textual
inputs into cross-modal contextualized semantics using a progressive
cross-attention interaction mechanism. These semantics are then injected into
the image encoder to enhance the detailed understanding of visual information.
To eliminate reliance on geometric prompts, we use a semantic prompting
strategy, replacing the original prompt encoder to sharpen the perception of
challenging targets. In addition, a similarity-sorting self-updating strategy
for memory and a mask-refining process is applied to further adapt to medical
imaging and enhance localized details. Comparative experiments conducted on
seven public datasets indicate that CRISP-SAM2 outperforms existing models.
Extensive analysis also demonstrates the effectiveness of our method, thereby
confirming its superior performance, especially in addressing the limitations
mentioned earlier. Our code is available at:
https://github.com/YU-deep/CRISP\_SAM2.git.
CRISP-SAM2, a novel model using cross-modal interaction and semantic prompting, enhances multi-organ medical segmentation by improving detail accuracy and reducing dependence on geometric prompts.
cross-modal interaction semantic prompting progressive cross-attention image encoder prompt encoder similarity-sorting self-updating mask-refining