Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for
  Efficient and Enhanced Video Reasoning
Despite advances in reinforcement learning (RL)-based video reasoning with
large language models (LLMs), data collection and finetuning remain significant
challenges. These methods often rely on large-scale supervised fine-tuning
(SFT) with extensive video data and long Chain-of-Thought (CoT) annotations,
making them costly and hard to scale. To address this, we present Video-RTS, a
new approach to improve video reasoning capability with drastically improved
data efficiency by combining data-efficient RL with a video-adaptive test-time
scaling (TTS) strategy. Based on observations about the data scaling of RL
samples, we skip the resource-intensive SFT step and employ efficient pure-RL
training with output-based rewards, requiring no additional annotations or
extensive fine-tuning. Furthermore, to utilize computational resources more
efficiently, we introduce a sparse-to-dense video TTS strategy that improves
inference by iteratively adding frames based on output consistency. We validate
our approach on multiple video reasoning benchmarks, showing that Video-RTS
surpasses existing video reasoning models by an average of 2.4% in accuracy
using only 3.6% training samples. For example, Video-RTS achieves a 4.2%
improvement on Video-Holmes, a recent and challenging video reasoning
benchmark, and a 2.6% improvement on MMVU. Notably, our pure RL training and
adaptive video TTS offer complementary strengths, enabling Video-RTS's strong
reasoning performance.
686fe3461de169176ba58b4e
Ziyang Wang
686fe3461de169176ba58b4f
Jaehong Yoon
686fe3461de169176ba58b50
Shoubin Yu
686fe3461de169176ba58b51
Md Mohaiminul Islam
686fe3461de169176ba58b52
Gedas Bertasius
686fe3461de169176ba58b53
Mohit Bansal
Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for
  Efficient and Enhanced Video Reasoning
Despite advances in reinforcement learning (RL)-based video reasoning with
large language models (LLMs), data collection and finetuning remain significant
challenges. These methods often rely on large-scale supervised fine-tuning
(SFT) with extensive video data and long Chain-of-Thought (CoT) annotations,
making them costly and hard to scale. To address this, we present Video-RTS, a
new approach to improve video reasoning capability with drastically improved
data efficiency by combining data-efficient RL with a video-adaptive test-time
scaling (TTS) strategy. Based on observations about the data scaling of RL
samples, we skip the resource-intensive SFT step and employ efficient pure-RL
training with output-based rewards, requiring no additional annotations or
extensive fine-tuning. Furthermore, to utilize computational resources more
efficiently, we introduce a sparse-to-dense video TTS strategy that improves
inference by iteratively adding frames based on output consistency. We validate
our approach on multiple video reasoning benchmarks, showing that Video-RTS
surpasses existing video reasoning models by an average of 2.4% in accuracy
using only 3.6% training samples. For example, Video-RTS achieves a 4.2%
improvement on Video-Holmes, a recent and challenging video reasoning
benchmark, and a 2.6% improvement on MMVU. Notably, our pure RL training and
adaptive video TTS offer complementary strengths, enabling Video-RTS's strong
reasoning performance.
Video-RTS enhances video reasoning efficiency by combining data-efficient RL and adaptive test-time scaling, achieving superior accuracy with minimal training data.
reinforcement learning large language models supervised fine-tuning Chain-of-Thought pure-RL training output-based rewards video-adaptive test-time scaling sparse-to-dense video TTS video reasoning benchmarks Video-Holmes MMVU