DINO-X: A Unified Vision Model for Open-World Object Detection and
  Understanding
In this paper, we introduce DINO-X, which is a unified object-centric vision
model developed by IDEA Research with the best open-world object detection
performance to date. DINO-X employs the same Transformer-based encoder-decoder
architecture as Grounding DINO 1.5 to pursue an object-level representation for
open-world object understanding. To make long-tailed object detection easy,
DINO-X extends its input options to support text prompt, visual prompt, and
customized prompt. With such flexible prompt options, we develop a universal
object prompt to support prompt-free open-world detection, making it possible
to detect anything in an image without requiring users to provide any prompt.
To enhance the model's core grounding capability, we have constructed a
large-scale dataset with over 100 million high-quality grounding samples,
referred to as Grounding-100M, for advancing the model's open-vocabulary
detection performance. Pre-training on such a large-scale grounding dataset
leads to a foundational object-level representation, which enables DINO-X to
integrate multiple perception heads to simultaneously support multiple object
perception and understanding tasks, including detection, segmentation, pose
estimation, object captioning, object-based QA, etc. Experimental results
demonstrate the superior performance of DINO-X. Specifically, the DINO-X Pro
model achieves 56.0 AP, 59.8 AP, and 52.4 AP on the COCO, LVIS-minival, and
LVIS-val zero-shot object detection benchmarks, respectively. Notably, it
scores 63.3 AP and 56.5 AP on the rare classes of LVIS-minival and LVIS-val
benchmarks, both improving the previous SOTA performance by 5.8 AP. Such a
result underscores its significantly improved capacity for recognizing
long-tailed objects.
6740b5bbb7c8a9b0ff8a9392
Tianhe Ren
6740b5bbb7c8a9b0ff8a9393
Yihao Chen
6740b5bbb7c8a9b0ff8a9394
Qing Jiang
6740b5bbb7c8a9b0ff8a9395
Zhaoyang Zeng
6740b5bbb7c8a9b0ff8a9396
Yuda Xiong
6740b5bbb7c8a9b0ff8a9397
Wenlong Liu
6740b5bbb7c8a9b0ff8a9398
Zhengyu Ma
6740b5bbb7c8a9b0ff8a9399
Junyi Shen
6740b5bbb7c8a9b0ff8a939a
Yuan Gao
6740b5bbb7c8a9b0ff8a939b
Xiaoke Jiang
6740b5bbb7c8a9b0ff8a939c
Xingyu Chen
6740b5bbb7c8a9b0ff8a939d
Zhuheng Song
6740b5bbb7c8a9b0ff8a939e
Yuhong Zhang
6740b5bbb7c8a9b0ff8a939f
Hongjie Huang
6740b5bbb7c8a9b0ff8a93a0
Han Gao
6740b5bbb7c8a9b0ff8a93a1
Shilong Liu
claimed_verified
2025-05-29T07:47:31.339Z
638b13c0c1d591879698f4e2
https://cdn-avatars.huggingface.co/v1/production/uploads/638b13c0c1d591879698f4e2/X8X4EWMXuzhBpG62wO2xS.jpeg
Shilong Liu
user
ShilongLiu
6740b5bbb7c8a9b0ff8a93a2
Hao Zhang
6740b5bbb7c8a9b0ff8a93a3
Feng Li
6740b5bbb7c8a9b0ff8a93a4
Kent Yu
6740b5bbb7c8a9b0ff8a93a5
Lei Zhang
DINO-X: A Unified Vision Model for Open-World Object Detection and
  Understanding
In this paper, we introduce DINO-X, which is a unified object-centric vision
model developed by IDEA Research with the best open-world object detection
performance to date. DINO-X employs the same Transformer-based encoder-decoder
architecture as Grounding DINO 1.5 to pursue an object-level representation for
open-world object understanding. To make long-tailed object detection easy,
DINO-X extends its input options to support text prompt, visual prompt, and
customized prompt. With such flexible prompt options, we develop a universal
object prompt to support prompt-free open-world detection, making it possible
to detect anything in an image without requiring users to provide any prompt.
To enhance the model's core grounding capability, we have constructed a
large-scale dataset with over 100 million high-quality grounding samples,
referred to as Grounding-100M, for advancing the model's open-vocabulary
detection performance. Pre-training on such a large-scale grounding dataset
leads to a foundational object-level representation, which enables DINO-X to
integrate multiple perception heads to simultaneously support multiple object
perception and understanding tasks, including detection, segmentation, pose
estimation, object captioning, object-based QA, etc. Experimental results
demonstrate the superior performance of DINO-X. Specifically, the DINO-X Pro
model achieves 56.0 AP, 59.8 AP, and 52.4 AP on the COCO, LVIS-minival, and
LVIS-val zero-shot object detection benchmarks, respectively. Notably, it
scores 63.3 AP and 56.5 AP on the rare classes of LVIS-minival and LVIS-val
benchmarks, both improving the previous SOTA performance by 5.8 AP. Such a
result underscores its significantly improved capacity for recognizing
long-tailed objects.
DINO-X, a Transformer-based object-centric vision model, achieves top performance in open-world object detection through flexible prompts, a large-scale dataset, and integration of multiple perception tasks.
Transformer-based encoder-decoder architecture open-world object detection text prompt visual prompt universal object prompt large-scale dataset Grounding-100M open-vocabulary detection object-level representation detection segmentation pose estimation object captioning object-based QA zero-shot object detection long-tailed objects