NaViL: Rethinking Scaling Properties of Native Multimodal Large Language
  Models under Data Constraints
Compositional training has been the de-facto paradigm in existing Multimodal
Large Language Models (MLLMs), where pre-trained vision encoders are connected
with pre-trained LLMs through continuous multimodal pre-training. However, the
multimodal scaling property of this paradigm remains difficult to explore due
to the separated training. In this paper, we focus on the native training of
MLLMs in an end-to-end manner and systematically study its design space and
scaling property under a practical setting, i.e., data constraint. Through
careful study of various choices in MLLM, we obtain the optimal
meta-architecture that best balances performance and training cost. After that,
we further explore the scaling properties of the native MLLM and indicate the
positively correlated scaling relationship between visual encoders and LLMs.
Based on these findings, we propose a native MLLM called NaViL, combined with a
simple and cost-effective recipe. Experimental results on 14 multimodal
benchmarks confirm the competitive performance of NaViL against existing MLLMs.
Besides that, our findings and results provide in-depth insights for the future
study of native MLLMs.
68e866db95e8e6771df38877
Changyao Tian
claimed_verified
2025-10-10T04:33:07.819Z
64b7475efa7eabaae5f7ba94
https://cdn-avatars.huggingface.co/v1/production/uploads/64b7475efa7eabaae5f7ba94/YLv44PZM6tw1sxACY1-U_.png
Changyao Tian
user
Changyao
68e866db95e8e6771df38878
Hao Li
68e866db95e8e6771df38879
Gen Luo
68e866db95e8e6771df3887a
Xizhou Zhu
68e866db95e8e6771df3887b
Weijie Su
68e866db95e8e6771df3887c
Hanming Deng
68e866db95e8e6771df3887d
Jinguo Zhu
68e866db95e8e6771df3887e
Jie Shao
68e866db95e8e6771df3887f
Ziran Zhu
68e866db95e8e6771df38880
Yunpeng Liu
68e866db95e8e6771df38881
Lewei Lu
68e866db95e8e6771df38882
Wenhai Wang
68e866db95e8e6771df38883
Hongsheng Li
68e866db95e8e6771df38884
Jifeng Dai
NaViL: Rethinking Scaling Properties of Native Multimodal Large Language
  Models under Data Constraints
Compositional training has been the de-facto paradigm in existing Multimodal
Large Language Models (MLLMs), where pre-trained vision encoders are connected
with pre-trained LLMs through continuous multimodal pre-training. However, the
multimodal scaling property of this paradigm remains difficult to explore due
to the separated training. In this paper, we focus on the native training of
MLLMs in an end-to-end manner and systematically study its design space and
scaling property under a practical setting, i.e., data constraint. Through
careful study of various choices in MLLM, we obtain the optimal
meta-architecture that best balances performance and training cost. After that,
we further explore the scaling properties of the native MLLM and indicate the
positively correlated scaling relationship between visual encoders and LLMs.
Based on these findings, we propose a native MLLM called NaViL, combined with a
simple and cost-effective recipe. Experimental results on 14 multimodal
benchmarks confirm the competitive performance of NaViL against existing MLLMs.
Besides that, our findings and results provide in-depth insights for the future
study of native MLLMs.
Native end-to-end training of Multimodal Large Language Models (MLLMs) achieves competitive performance with a balanced design and scaling relationship between visual encoders and LLMs.
Multimodal Large Language Models compositional training continuous multimodal pre-training end-to-end training meta-architecture scaling property visual encoders LLMs NaViL