Unified Reinforcement and Imitation Learning for Vision-Language Models
Vision-Language Models (VLMs) have achieved remarkable progress, yet their
large scale often renders them impractical for resource-constrained
environments. This paper introduces Unified Reinforcement and Imitation
Learning (RIL), a novel and efficient training algorithm designed to create
powerful, lightweight VLMs. RIL distinctively combines the strengths of
reinforcement learning with adversarial imitation learning. This enables
smaller student VLMs not only to mimic the sophisticated text generation of
large teacher models but also to systematically improve their generative
capabilities through reinforcement signals. Key to our imitation framework is
an LLM-based discriminator that adeptly distinguishes between student and
teacher outputs, complemented by guidance from multiple large teacher VLMs to
ensure diverse learning. This unified learning strategy, leveraging both
reinforcement and imitation, empowers student models to achieve significant
performance gains, making them competitive with leading closed-source VLMs.
Extensive experiments on diverse vision-language benchmarks demonstrate that
RIL significantly narrows the performance gap with state-of-the-art open- and
closed-source VLMs and, in several instances, surpasses them.
68f9929eb9b2e4ae04673817
Byung-Kwan Lee
claimed_verified
2025-10-23T14:48:24.236Z
657152eb12f162153b50ec9d
https://cdn-avatars.huggingface.co/v1/production/uploads/657152eb12f162153b50ec9d/qnldHP35PclV0pDz_05q8.jpeg
Byung-Kwan Lee
user
BK-Lee
68f9929eb9b2e4ae04673818
Ryo Hachiuma
68f9929eb9b2e4ae04673819
Yong Man Ro
68f9929eb9b2e4ae0467381a
Yu-Chiang Frank Wang
68f9929eb9b2e4ae0467381b
Yueh-Hua Wu
Unified Reinforcement and Imitation Learning for Vision-Language Models
Vision-Language Models (VLMs) have achieved remarkable progress, yet their
large scale often renders them impractical for resource-constrained
environments. This paper introduces Unified Reinforcement and Imitation
Learning (RIL), a novel and efficient training algorithm designed to create
powerful, lightweight VLMs. RIL distinctively combines the strengths of
reinforcement learning with adversarial imitation learning. This enables
smaller student VLMs not only to mimic the sophisticated text generation of
large teacher models but also to systematically improve their generative
capabilities through reinforcement signals. Key to our imitation framework is
an LLM-based discriminator that adeptly distinguishes between student and
teacher outputs, complemented by guidance from multiple large teacher VLMs to
ensure diverse learning. This unified learning strategy, leveraging both
reinforcement and imitation, empowers student models to achieve significant
performance gains, making them competitive with leading closed-source VLMs.
Extensive experiments on diverse vision-language benchmarks demonstrate that
RIL significantly narrows the performance gap with state-of-the-art open- and
closed-source VLMs and, in several instances, surpasses them.
A unified reinforcement and imitation learning algorithm creates efficient, lightweight vision-language models that match or exceed leading VLMs in performance.
Unified Reinforcement and Imitation Learning (RIL) reinforcement learning adversarial imitation learning LLM-based discriminator vision-language models (VLMs) text generation generative capabilities vision-language benchmarks