InstaGen: Enhancing Object Detection by Training on Synthetic Dataset
In this paper, we introduce a novel paradigm to enhance the ability of object
detector, e.g., expanding categories or improving detection performance, by
training on synthetic dataset generated from diffusion models. Specifically, we
integrate an instance-level grounding head into a pre-trained, generative
diffusion model, to augment it with the ability of localising arbitrary
instances in the generated images. The grounding head is trained to align the
text embedding of category names with the regional visual feature of the
diffusion model, using supervision from an off-the-shelf object detector, and a
novel self-training scheme on (novel) categories not covered by the detector.
This enhanced version of diffusion model, termed as InstaGen, can serve as a
data synthesizer for object detection. We conduct thorough experiments to show
that, object detector can be enhanced while training on the synthetic dataset
from InstaGen, demonstrating superior performance over existing
state-of-the-art methods in open-vocabulary (+4.5 AP) and data-sparse (+1.2 to
5.2 AP) scenarios.
65c5a386eabb2d3066caf282
Chengjian Feng
admin_assigned
2024-02-09T10:07:54.572Z
639c3582e62c431aa8d9028e
/avatars/b23c73d0b7ea062f5292c5838b83555a.svg
Chengjian Feng
user
jian223
65c5a386eabb2d3066caf283
Yujie Zhong
admin_assigned
2024-02-09T10:08:01.808Z
64b4adc90a54158d663268a5
/avatars/885b8c05d070c69f28879290c33c0aa7.svg
Yujie Zhong
user
zhongyujie
65c5a386eabb2d3066caf284
Zequn Jie
65c5a386eabb2d3066caf285
Weidi Xie
admin_assigned
2024-02-09T10:08:26.940Z
63f1841f2f7c0152e8748ec4
/avatars/5398658c7d0cf556531a625a4ca5d18a.svg
Xie
user
Weidi
65c5a386eabb2d3066caf286
Lin Ma
InstaGen: Enhancing Object Detection by Training on Synthetic Dataset
In this paper, we introduce a novel paradigm to enhance the ability of object
detector, e.g., expanding categories or improving detection performance, by
training on synthetic dataset generated from diffusion models. Specifically, we
integrate an instance-level grounding head into a pre-trained, generative
diffusion model, to augment it with the ability of localising arbitrary
instances in the generated images. The grounding head is trained to align the
text embedding of category names with the regional visual feature of the
diffusion model, using supervision from an off-the-shelf object detector, and a
novel self-training scheme on (novel) categories not covered by the detector.
This enhanced version of diffusion model, termed as InstaGen, can serve as a
data synthesizer for object detection. We conduct thorough experiments to show
that, object detector can be enhanced while training on the synthetic dataset
from InstaGen, demonstrating superior performance over existing
state-of-the-art methods in open-vocabulary (+4.5 AP) and data-sparse (+1.2 to
5.2 AP) scenarios.
A diffusion model enhanced with an instance-level grounding head serves as a synthetic data generator to improve object detection performance, especially in open-vocabulary and data-sparse scenarios.
diffusion models instance-level grounding text embedding regional visual feature self-training InstaGen object detection synthetic dataset open-vocabulary data-sparse