Region-based Cluster Discrimination for Visual Representation Learning
Learning visual representations is foundational for a broad spectrum of
downstream tasks. Although recent vision-language contrastive models, such as
CLIP and SigLIP, have achieved impressive zero-shot performance via large-scale
vision-language alignment, their reliance on global representations constrains
their effectiveness for dense prediction tasks, such as grounding, OCR, and
segmentation. To address this gap, we introduce Region-Aware Cluster
Discrimination (RICE), a novel method that enhances region-level visual and OCR
capabilities. We first construct a billion-scale candidate region dataset and
propose a Region Transformer layer to extract rich regional semantics. We
further design a unified region cluster discrimination loss that jointly
supports object and OCR learning within a single classification framework,
enabling efficient and scalable distributed training on large-scale data.
Extensive experiments show that RICE consistently outperforms previous methods
on tasks, including segmentation, dense detection, and visual perception for
Multimodal Large Language Models (MLLMs). The pre-trained models have been
released at https://github.com/deepglint/MVT.
68883088af872d625c10c5eb
Yin Xie
68883088af872d625c10c5ec
Kaicheng Yang
claimed_verified
2025-07-29T07:08:26.917Z
63e202f352b7578dba448ab5
https://cdn-avatars.huggingface.co/v1/production/uploads/63e202f352b7578dba448ab5/8itVBLcv14m7OVsoF8h1o.jpeg
Yang
user
Kaichengalex
68883088af872d625c10c5ed
Xiang An
claimed_verified
2025-07-29T07:08:29.232Z
6478679d7b370854241b2ad8
https://cdn-avatars.huggingface.co/v1/production/uploads/6478679d7b370854241b2ad8/dBczWYYdfEt9tQcnVGhQk.jpeg
xiangan
user
xiangan
68883088af872d625c10c5ee
Kun Wu
claimed_verified
2025-07-29T07:08:24.529Z
64f597588b6d053c709debd9
/avatars/85d046a412c503f883e16324030a457b.svg
Kun
user
Athinklo
68883088af872d625c10c5ef
Yongle Zhao
claimed_verified
2025-09-05T14:17:13.525Z
6433e9e4ea46c00990460ab0
/avatars/3a4bec74eeec94713e8eb5e290d00765.svg
yonglezhao
user
yonglezhao
68883088af872d625c10c5f0
Weimo Deng
68883088af872d625c10c5f1
Zimin Ran
68883088af872d625c10c5f2
Yumeng Wang
68883088af872d625c10c5f3
Ziyong Feng
68883088af872d625c10c5f4
Roy Miles
claimed_verified
2025-09-12T16:22:55.805Z
6478cea13b7f8b1f624997e5
/avatars/34641d634da565cd50a7310de3f4513f.svg
Roy Miles
user
iyop45
68883088af872d625c10c5f5
Ismail Elezi
68883088af872d625c10c5f6
Jiankang Deng
Region-based Cluster Discrimination for Visual Representation Learning
Learning visual representations is foundational for a broad spectrum of
downstream tasks. Although recent vision-language contrastive models, such as
CLIP and SigLIP, have achieved impressive zero-shot performance via large-scale
vision-language alignment, their reliance on global representations constrains
their effectiveness for dense prediction tasks, such as grounding, OCR, and
segmentation. To address this gap, we introduce Region-Aware Cluster
Discrimination (RICE), a novel method that enhances region-level visual and OCR
capabilities. We first construct a billion-scale candidate region dataset and
propose a Region Transformer layer to extract rich regional semantics. We
further design a unified region cluster discrimination loss that jointly
supports object and OCR learning within a single classification framework,
enabling efficient and scalable distributed training on large-scale data.
Extensive experiments show that RICE consistently outperforms previous methods
on tasks, including segmentation, dense detection, and visual perception for
Multimodal Large Language Models (MLLMs). The pre-trained models have been
released at https://github.com/deepglint/MVT.
RICE, a novel method using a Region Transformer and region cluster discrimination loss, enhances region-level visual and OCR capabilities, outperforming previous methods in tasks like segmentation and dense detection.
Region-Aware Cluster Discrimination RICE Region Transformer region cluster discrimination loss dense prediction tasks grounding OCR segmentation dense detection Multimodal Large Language Models MLLMs