CLIPSym: Delving into Symmetry Detection with CLIP
Symmetry is one of the most fundamental geometric cues in computer vision,
and detecting it has been an ongoing challenge. With the recent advances in
vision-language models,~i.e., CLIP, we investigate whether a pre-trained CLIP
model can aid symmetry detection by leveraging the additional symmetry cues
found in the natural image descriptions. We propose CLIPSym, which leverages
CLIP's image and language encoders and a rotation-equivariant decoder based on
a hybrid of Transformer and G-Convolution to detect rotation and reflection
symmetries. To fully utilize CLIP's language encoder, we have developed a novel
prompting technique called Semantic-Aware Prompt Grouping (SAPG), which
aggregates a diverse set of frequent object-based prompts to better integrate
the semantic cues for symmetry detection. Empirically, we show that CLIPSym
outperforms the current state-of-the-art on three standard symmetry detection
datasets (DENDI, SDRW, and LDRS). Finally, we conduct detailed ablations
verifying the benefits of CLIP's pre-training, the proposed equivariant
decoder, and the SAPG technique. The code is available at
https://github.com/timyoung2333/CLIPSym.
68b53dcf851c6e7b001ecad7
Tinghan Yang
68b53dcf851c6e7b001ecad8
Md Ashiqur Rahman
claimed_verified
2025-09-24T13:57:45.677Z
661e07e02a8496916011c08a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/6vIkL4LJ0bLeJL99E20F_.jpeg
Md Ashiqur Rahman
user
ashiq24
68b53dcf851c6e7b001ecad9
Raymond A. Yeh
CLIPSym: Delving into Symmetry Detection with CLIP
Symmetry is one of the most fundamental geometric cues in computer vision,
and detecting it has been an ongoing challenge. With the recent advances in
vision-language models,~i.e., CLIP, we investigate whether a pre-trained CLIP
model can aid symmetry detection by leveraging the additional symmetry cues
found in the natural image descriptions. We propose CLIPSym, which leverages
CLIP's image and language encoders and a rotation-equivariant decoder based on
a hybrid of Transformer and G-Convolution to detect rotation and reflection
symmetries. To fully utilize CLIP's language encoder, we have developed a novel
prompting technique called Semantic-Aware Prompt Grouping (SAPG), which
aggregates a diverse set of frequent object-based prompts to better integrate
the semantic cues for symmetry detection. Empirically, we show that CLIPSym
outperforms the current state-of-the-art on three standard symmetry detection
datasets (DENDI, SDRW, and LDRS). Finally, we conduct detailed ablations
verifying the benefits of CLIP's pre-training, the proposed equivariant
decoder, and the SAPG technique. The code is available at
https://github.com/timyoung2333/CLIPSym.
CLIPSym, a vision-language model using CLIP, enhances symmetry detection through a rotation-equivariant decoder and semantic-aware prompting, outperforming existing methods on standard datasets.
CLIP CLIPSym rotation-equivariant decoder Transformer G-Convolution Semantic-Aware Prompt Grouping SAPG symmetry detection DENDI SDRW LDRS