MM-LLMs: Recent Advances in MultiModal Large Language Models
In the past year, MultiModal Large Language Models (MM-LLMs) have undergone
substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or
outputs via cost-effective training strategies. The resulting models not only
preserve the inherent reasoning and decision-making capabilities of LLMs but
also empower a diverse range of MM tasks. In this paper, we provide a
comprehensive survey aimed at facilitating further research of MM-LLMs.
Specifically, we first outline general design formulations for model
architecture and training pipeline. Subsequently, we provide brief
introductions of 26 existing MM-LLMs, each characterized by its specific
formulations. Additionally, we review the performance of MM-LLMs on mainstream
benchmarks and summarize key training recipes to enhance the potency of
MM-LLMs. Lastly, we explore promising directions for MM-LLMs while concurrently
maintaining a real-time tracking website for the latest developments in the
field. We hope that this survey contributes to the ongoing advancement of the
MM-LLMs domain.
65b1ea38ce74220b8315682e
Duzhen Zhang
claimed_verified
2025-07-18T07:51:26.387Z
662383a20edeabfe3b64a6a5
/avatars/a76da726002d853dd08a51a6af6311d9.svg
Duzhen Zhang
user
ShowerMaker
65b1ea38ce74220b8315682f
Yahan Yu
claimed_verified
2024-02-27T13:49:13.051Z
65ddd02438ad9c41e2b1f54f
/avatars/dc584e49e26d2fa3c5bca5c90aeca913.svg
Yahan Yu
user
YahanYu
65b1ea38ce74220b83156830
Chenxing Li
65b1ea38ce74220b83156831
Jiahua Dong
65b1ea38ce74220b83156832
Dan Su
claimed_verified
2024-01-25T08:03:57.150Z
64c8fa55dc59e43a6100b382
/avatars/0a4381fbab27e2ae2438c275a7046f43.svg
dan su
user
sudanenator
65b1ea38ce74220b83156833
Chenhui Chu
65b1ea38ce74220b83156834
Dong Yu
MM-LLMs: Recent Advances in MultiModal Large Language Models
In the past year, MultiModal Large Language Models (MM-LLMs) have undergone
substantial advancements, augmenting off-the-shelf LLMs to support MM inputs or
outputs via cost-effective training strategies. The resulting models not only
preserve the inherent reasoning and decision-making capabilities of LLMs but
also empower a diverse range of MM tasks. In this paper, we provide a
comprehensive survey aimed at facilitating further research of MM-LLMs.
Specifically, we first outline general design formulations for model
architecture and training pipeline. Subsequently, we provide brief
introductions of 26 existing MM-LLMs, each characterized by its specific
formulations. Additionally, we review the performance of MM-LLMs on mainstream
benchmarks and summarize key training recipes to enhance the potency of
MM-LLMs. Lastly, we explore promising directions for MM-LLMs while concurrently
maintaining a real-time tracking website for the latest developments in the
field. We hope that this survey contributes to the ongoing advancement of the
MM-LLMs domain.
A survey examining the design, performance, and future directions of MultiModal Large Language Models (MM-LLMs) to enhance reasoning and MM task capabilities.
MultiModal Large Language Models MM-LLMs model architecture training pipeline existing MM-LLMs mainstream benchmarks training recipes