Scaling RL to Long Videos
We introduce a full-stack framework that scales up reasoning in
vision-language models (VLMs) to long videos, leveraging reinforcement
learning. We address the unique challenges of long video reasoning by
integrating three critical components: (1) a large-scale dataset,
LongVideo-Reason, comprising 52K long video QA pairs with high-quality
reasoning annotations across diverse domains such as sports, games, and vlogs;
(2) a two-stage training pipeline that extends VLMs with chain-of-thought
supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a
training infrastructure for long video RL, named Multi-modal Reinforcement
Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a
vLLM-based engine tailored for long video, using cached video embeddings for
efficient rollout and prefilling. In experiments, LongVILA-R1-7B achieves
strong performance on long video QA benchmarks such as VideoMME. It also
outperforms Video-R1-7B and even matches Gemini-1.5-Pro across temporal
reasoning, goal and purpose reasoning, spatial reasoning, and plot reasoning on
our LongVideo-Reason-eval benchmark. Notably, our MR-SP system achieves up to
2.1x speedup on long video RL training. LongVILA-R1 demonstrates consistent
performance gains as the number of input video frames scales. LongVILA-R1 marks
a firm step towards long video reasoning in VLMs. In addition, we release our
training system for public availability that supports RL training on various
modalities (video, text, and audio), various models (VILA and Qwen series), and
even image and video generation models. On a single A100 node (8 GPUs), it
supports RL training on hour-long videos (e.g., 3,600 frames / around 256k
tokens).
68706bdcc8391850d60977eb
Yukang Chen
68706bdcc8391850d60977ec
Wei Huang
claimed_verified
2025-07-13T07:26:25.046Z
656db3f53dc1d277e5a64410
https://cdn-avatars.huggingface.co/v1/production/uploads/656db3f53dc1d277e5a64410/9kiY2K3MCRcBDk7MrkTBK.png
Wei Huang
user
AaronHuangWei
68706bdcc8391850d60977ed
Baifeng Shi
claimed_verified
2025-07-28T08:46:36.249Z
649004218f7cbbc94c782db6
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/AdgLVfAIpWlug4jXTaEK-.jpeg
Baifeng Shi
user
bfshi
68706bdcc8391850d60977ee
Qinghao Hu
claimed_verified
2025-07-13T07:26:27.276Z
62fb8412610dae1bcd036db7
https://cdn-avatars.huggingface.co/v1/production/uploads/62fb8412610dae1bcd036db7/LIaEoDrvrq8hDl8509ZuF.jpeg
Qinghao Hu
user
Qinghao
68706bdcc8391850d60977ef
Hanrong Ye
68706bdcc8391850d60977f0
Ligeng Zhu
68706bdcc8391850d60977f1
Zhijian Liu
claimed_verified
2025-07-21T08:30:36.232Z
650dac79b959b0e1d41d7378
https://cdn-avatars.huggingface.co/v1/production/uploads/650dac79b959b0e1d41d7378/mzbN0MFk3k8b94FQ40I7L.jpeg
Zhijian Liu
user
zhijianliu
68706bdcc8391850d60977f2
Pavlo Molchanov
68706bdcc8391850d60977f3
Jan Kautz
68706bdcc8391850d60977f4
Xiaojuan Qi
claimed_verified
2025-07-15T19:11:06.027Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
68706bdcc8391850d60977f5
Sifei Liu
68706bdcc8391850d60977f6
Hongxu Yin
68706bdcc8391850d60977f7
Yao Lu
68706bdcc8391850d60977f8
Song Han
Scaling RL to Long Videos
We introduce a full-stack framework that scales up reasoning in
vision-language models (VLMs) to long videos, leveraging reinforcement
learning. We address the unique challenges of long video reasoning by
integrating three critical components: (1) a large-scale dataset,
LongVideo-Reason, comprising 52K long video QA pairs with high-quality
reasoning annotations across diverse domains such as sports, games, and vlogs;
(2) a two-stage training pipeline that extends VLMs with chain-of-thought
supervised fine-tuning (CoT-SFT) and reinforcement learning (RL); and (3) a
training infrastructure for long video RL, named Multi-modal Reinforcement
Sequence Parallelism (MR-SP), which incorporates sequence parallelism and a
vLLM-based engine tailored for long video, using cached video embeddings for
efficient rollout and prefilling. In experiments, LongVILA-R1-7B achieves
strong performance on long video QA benchmarks such as VideoMME. It also
outperforms Video-R1-7B and even matches Gemini-1.5-Pro across temporal
reasoning, goal and purpose reasoning, spatial reasoning, and plot reasoning on
our LongVideo-Reason-eval benchmark. Notably, our MR-SP system achieves up to
2.1x speedup on long video RL training. LongVILA-R1 demonstrates consistent
performance gains as the number of input video frames scales. LongVILA-R1 marks
a firm step towards long video reasoning in VLMs. In addition, we release our
training system for public availability that supports RL training on various
modalities (video, text, and audio), various models (VILA and Qwen series), and
even image and video generation models. On a single A100 node (8 GPUs), it
supports RL training on hour-long videos (e.g., 3,600 frames / around 256k
tokens).
A framework scales vision-language models for long video reasoning using reinforcement learning, achieving strong performance on benchmarks and demonstrating consistent gains with increased video frames.
vision-language models reinforcement learning LongVideo-Reason chain-of-thought supervised fine-tuning CoT-SFT Multi-modal Reinforcement Sequence Parallelism MR-SP sequence parallelism vLLM VideoMME LongVideo-Reason-eval temporal reasoning goal and purpose reasoning spatial reasoning plot reasoning LongVILA-R1-7B Video-R1-7B Gemini-1.5-Pro image and video generation models