Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
  Reinforcement
Traditional methods for reasoning segmentation rely on supervised fine-tuning
with categorical labels and simple descriptions, limiting its out-of-domain
generalization and lacking explicit reasoning processes. To address these
limitations, we propose Seg-Zero, a novel framework that demonstrates
remarkable generalizability and derives explicit chain-of-thought reasoning
through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
consisting of a reasoning model and a segmentation model. The reasoning model
interprets user intentions, generates explicit reasoning chains, and produces
positional prompts, which are subsequently used by the segmentation model to
generate precious pixel-level masks. We design a sophisticated reward mechanism
that integrates both format and accuracy rewards to effectively guide
optimization directions. Trained exclusively via reinforcement learning with
GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
generalization and exhibits emergent test-time reasoning capabilities.
Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
improvement highlights Seg-Zero's ability to generalize across domains while
presenting an explicit reasoning process. Code is available at
https://github.com/dvlab-research/Seg-Zero.
67cf990ca80a73999cc816c3
Yuqi Liu
67cf990ca80a73999cc816c4
Bohao Peng
admin_assigned
2025-03-11T14:51:05.435Z
673a10f911b7efeeedabc252
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/T7ySn7F0pTVCvRdcvMz3d.png
Bohao Peng
user
BoHao0326
67cf990ca80a73999cc816c5
Zhisheng Zhong
admin_assigned
2025-03-11T14:50:58.642Z
65d882d30f35ed3f52d3ae2c
/avatars/22cda67c3fcd7150320ec3551eda90f5.svg
Zhisheng Zhong
user
zszhong
67cf990ca80a73999cc816c6
Zihao Yue
claimed_verified
2025-05-13T08:13:34.470Z
6517f0df593b3af3120d242e
https://cdn-avatars.huggingface.co/v1/production/uploads/6517f0df593b3af3120d242e/gx4G616pnhw2ugEDPjwnH.jpeg
Zihao Yue
user
yuezih
67cf990ca80a73999cc816c7
Fanbin Lu
admin_assigned
2025-03-11T14:51:21.227Z
645b6094bc7518912e1fbc34
/avatars/5e1a875ba3ce350e71fe7049ca6a44c1.svg
Fanbin Lu
user
Fanbin
67cf990ca80a73999cc816c8
Bei Yu
67cf990ca80a73999cc816c9
Jiaya Jia
Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
  Reinforcement
Traditional methods for reasoning segmentation rely on supervised fine-tuning
with categorical labels and simple descriptions, limiting its out-of-domain
generalization and lacking explicit reasoning processes. To address these
limitations, we propose Seg-Zero, a novel framework that demonstrates
remarkable generalizability and derives explicit chain-of-thought reasoning
through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
consisting of a reasoning model and a segmentation model. The reasoning model
interprets user intentions, generates explicit reasoning chains, and produces
positional prompts, which are subsequently used by the segmentation model to
generate precious pixel-level masks. We design a sophisticated reward mechanism
that integrates both format and accuracy rewards to effectively guide
optimization directions. Trained exclusively via reinforcement learning with
GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
generalization and exhibits emergent test-time reasoning capabilities.
Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
improvement highlights Seg-Zero's ability to generalize across domains while
presenting an explicit reasoning process. Code is available at
https://github.com/dvlab-research/Seg-Zero.
Seg-Zero, a novel framework using cognitive reinforcement, achieves robust zero-shot generalization in segmentation with explicit reasoning capabilities.
reasoning segmentation supervised fine-tuning categorical labels explicit reasoning processes decoupled architecture reasoning model segmentation model positional prompts cognitive reinforcement reward mechanism reinforcement learning GRPO zero-shot performance ReasonSeg benchmark