Infinity-MM: Scaling Multimodal Performance with Large-Scale and
  High-Quality Instruction Data
Vision-Language Models (VLMs) have recently made significant progress, but
the limited scale and quality of open-source instruction data hinder their
performance compared to closed-source models. In this work, we address this
limitation by introducing Infinity-MM, a large-scale multimodal instruction
dataset with 40 million samples, enhanced through rigorous quality filtering
and deduplication. We also propose a synthetic instruction generation method
based on open-source VLMs, using detailed image annotations and diverse
question generation. Using this data, we trained a 2-billion-parameter VLM,
Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of
similar scale. This demonstrates that expanding instruction data and generating
synthetic data can significantly improve the performance of open-source models.
671afdd1ffcdeeb655cbe21b
Shuhao Gu
admin_assigned
2024-10-28T15:19:19.039Z
642e72cec1b0f8e4e76af16d
/avatars/f900811d3c22a114c67283b646949f86.svg
shuhao gu
user
gsh33
671afdd1ffcdeeb655cbe21c
Jialing Zhang
claimed_verified
2024-11-15T09:42:08.927Z
65473b574486732720548dae
/avatars/180ff1bd6a58b87d702c411c2df6b663.svg
Jialing Zhang
user
JialingZhang
671afdd1ffcdeeb655cbe21d
Siyuan Zhou
claimed_verified
2024-10-30T09:54:50.318Z
663e2aef881fe6a203c681e8
/avatars/4bcdd7db684cf98e6881f1ee29b2e16c.svg
Siyuan Zhou
user
Ssyuan9876
671afdd1ffcdeeb655cbe21e
Kevin Yu
671afdd1ffcdeeb655cbe21f
Zhaohu Xing
admin_assigned
2024-10-28T15:20:13.818Z
632c2100ea6e62428ab201e9
/avatars/5bd355e095af93261928198e3d6d5696.svg
Xingzhaohu
user
xingzhaohu
671afdd1ffcdeeb655cbe220
Liangdong Wang
claimed_verified
2024-10-25T09:30:05.708Z
63a11ce02fabbbb899a01d58
/avatars/ee3d4088b6d32b2c18b8be91913e90dd.svg
ldwang
user
ldwang
671afdd1ffcdeeb655cbe221
Zhou Cao
claimed_verified
2025-07-08T09:05:56.548Z
66aadfc1344279e0243d4569
/avatars/d2afbceb2e6279e42ed9ad98dffa7f0a.svg
Caozhou
user
Caozhou1995
671afdd1ffcdeeb655cbe222
Jintao Jia
admin_assigned
2024-10-28T15:20:22.639Z
6695dc41b91490da0249797e
/avatars/59df94f34fb4967e39b8237aac669546.svg
Jintao Jia
user
jjtnb
671afdd1ffcdeeb655cbe223
Zhuoyi Zhang
671afdd1ffcdeeb655cbe224
Yixuan Wang
claimed_verified
2024-10-28T16:55:00.224Z
66a0a5721bb371390c70f5a5
/avatars/9403480ef1277ac789ad2d375f398113.svg
Grace Wang
user
Gracecc
671afdd1ffcdeeb655cbe225
Zhenchong Hu
admin_assigned
2024-10-28T15:21:14.573Z
669600f8cebc15b84aff0337
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/N755P9kTEM4Vu_UUY_lAU.jpeg
胡振崇
user
zhenchonghu
671afdd1ffcdeeb655cbe226
Bo-Wen Zhang
claimed_verified
2024-10-28T11:04:53.176Z
6335113375bed9932474315e
/avatars/aa1837a3a07514b89c48de888934a7b2.svg
bowenzhang
user
bowen92
671afdd1ffcdeeb655cbe227
Jijie Li
admin_assigned
2024-10-28T15:21:27.092Z
65e00e64092e556a9a7dd429
/avatars/57959174ef1b0a9dbb14e53d4aea58a8.svg
lijijie
user
Avoczh
671afdd1ffcdeeb655cbe228
Dong Liang
671afdd1ffcdeeb655cbe229
Yingli Zhao
671afdd1ffcdeeb655cbe22a
Yulong Ao
admin_assigned
2024-10-28T15:22:10.950Z
64f3eb4fd41e83d74a0173ec
/avatars/461f804bb5a7c5b31f862cfa27e52de1.svg
Yulong Ao
user
aoyulong
671afdd1ffcdeeb655cbe22b
Yaoqi Liu
671afdd1ffcdeeb655cbe22c
Fangxiang Feng
671afdd1ffcdeeb655cbe22d
Guang Liu
claimed_verified
2024-10-28T11:22:42.703Z
632c234f42c386ebd2710434
https://cdn-avatars.huggingface.co/v1/production/uploads/632c234f42c386ebd2710434/HyWRWi063S69JTy_IMjoe.jpeg
Guang Liu
user
ZacLiu
Infinity-MM: Scaling Multimodal Performance with Large-Scale and
  High-Quality Instruction Data
Vision-Language Models (VLMs) have recently made significant progress, but
the limited scale and quality of open-source instruction data hinder their
performance compared to closed-source models. In this work, we address this
limitation by introducing Infinity-MM, a large-scale multimodal instruction
dataset with 40 million samples, enhanced through rigorous quality filtering
and deduplication. We also propose a synthetic instruction generation method
based on open-source VLMs, using detailed image annotations and diverse
question generation. Using this data, we trained a 2-billion-parameter VLM,
Aquila-VL-2B, achieving state-of-the-art (SOTA) performance for models of
similar scale. This demonstrates that expanding instruction data and generating
synthetic data can significantly improve the performance of open-source models.
Infinity-MM, a large-scale multimodal instruction dataset, enhances open-source VLMs through rigorous filtering and synthetic data generation, achieving state-of-the-art performance in models of similar size.
Vision-Language Models multimodal instruction dataset quality filtering deduplication synthetic instruction generation Aquila-VL-2B state-of-the-art performance