Gen2Det: Generate to Detect
Recently diffusion models have shown improvement in synthetic image quality
as well as better control in generation. We motivate and present Gen2Det, a
simple modular pipeline to create synthetic training data for object detection
for free by leveraging state-of-the-art grounded image generation methods.
Unlike existing works which generate individual object instances, require
identifying foreground followed by pasting on other images, we simplify to
directly generating scene-centric images. In addition to the synthetic data,
Gen2Det also proposes a suite of techniques to best utilize the generated data,
including image-level filtering, instance-level filtering, and better training
recipe to account for imperfections in the generation. Using Gen2Det, we show
healthy improvements on object detection and segmentation tasks under various
settings and agnostic to detection methods. In the long-tailed detection
setting on LVIS, Gen2Det improves the performance on rare categories by a large
margin while also significantly improving the performance on other categories,
e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training
on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO,
Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In
the most general detection setting, Gen2Det still demonstrates robust
performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and
0.32 points.
65728dfaca0139e4acc23821
Saksham Suri
admin_assigned
2023-12-08T10:39:38.885Z
645417e98617183806210cfc
/avatars/7a3ba5b0c38878db51da7091fed15f99.svg
Saksham Suri
user
sakshams
65728dfaca0139e4acc23822
Fanyi Xiao
admin_assigned
2023-12-08T10:39:47.351Z
641cf83645487810d136cb34
/avatars/95bfc200f86abbe9e8df0308517a9ce4.svg
Fanyi Xiao
user
fanyix
65728dfaca0139e4acc23823
Animesh Sinha
admin_assigned
2023-12-08T10:39:53.325Z
63772d28e069bdcdbb8e4a29
/avatars/be71b00e009f2093a3ff79f07815e68e.svg
Animesh Sinha
user
animeshsinha
65728dfaca0139e4acc23824
Sean Chang Culatana
65728dfaca0139e4acc23825
Raghuraman Krishnamoorthi
65728dfaca0139e4acc23826
Chenchen Zhu
65728dfaca0139e4acc23827
Abhinav Shrivastava
admin_assigned
2023-12-08T10:40:43.124Z
64b683acb20146a0d307222c
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/l665j1WeROwhQKgIsszYp.jpeg
Abhinav Shrivastava
user
abhi2610
Gen2Det: Generate to Detect
Recently diffusion models have shown improvement in synthetic image quality
as well as better control in generation. We motivate and present Gen2Det, a
simple modular pipeline to create synthetic training data for object detection
for free by leveraging state-of-the-art grounded image generation methods.
Unlike existing works which generate individual object instances, require
identifying foreground followed by pasting on other images, we simplify to
directly generating scene-centric images. In addition to the synthetic data,
Gen2Det also proposes a suite of techniques to best utilize the generated data,
including image-level filtering, instance-level filtering, and better training
recipe to account for imperfections in the generation. Using Gen2Det, we show
healthy improvements on object detection and segmentation tasks under various
settings and agnostic to detection methods. In the long-tailed detection
setting on LVIS, Gen2Det improves the performance on rare categories by a large
margin while also significantly improving the performance on other categories,
e.g. we see an improvement of 2.13 Box AP and 1.84 Mask AP over just training
on real data on LVIS with Mask R-CNN. In the low-data regime setting on COCO,
Gen2Det consistently improves both Box and Mask AP by 2.27 and 1.85 points. In
the most general detection setting, Gen2Det still demonstrates robust
performance gains, e.g. it improves the Box and Mask AP on COCO by 0.45 and
0.32 points.
Gen2Det generates scene-centric synthetic training data for object detection using grounded image generation and improves performance across various settings.
diffusion models scene-centric images image-level filtering instance-level filtering training recipe long-tailed detection low-data regime Box AP Mask AP LVIS Mask R-CNN COCO