Exploring the Effect of Reinforcement Learning on Video Understanding:
  Insights from SEED-Bench-R1
Recent advancements in Chain of Thought (COT) generation have significantly
improved the reasoning capabilities of Large Language Models (LLMs), with
reinforcement learning (RL) emerging as an effective post-training approach.
Multimodal Large Language Models (MLLMs) inherit this reasoning potential but
remain underexplored in tasks requiring both perception and logical reasoning.
To address this, we introduce SEED-Bench-R1, a benchmark designed to
systematically evaluate post-training methods for MLLMs in video understanding.
It includes intricate real-world videos and complex everyday planning tasks in
the format of multiple-choice questions, requiring sophisticated perception and
reasoning. SEED-Bench-R1 assesses generalization through a three-level
hierarchy: in-distribution, cross-environment, and cross-environment-task
scenarios, equipped with a large-scale training dataset with easily verifiable
ground-truth answers. Using Qwen2-VL-Instruct-7B as a base model, we compare RL
with supervised fine-tuning (SFT), demonstrating RL's data efficiency and
superior performance on both in-distribution and out-of-distribution tasks,
even outperforming SFT on general video understanding benchmarks like
LongVideoBench. Our detailed analysis reveals that RL enhances visual
perception but often produces less logically coherent reasoning chains. We
identify key limitations such as inconsistent reasoning and overlooked visual
cues, and suggest future improvements in base model reasoning, reward modeling,
and RL robustness against noisy signals.
67eca2b8351721d62aa537df
Yi Chen
claimed_verified
2025-04-02T08:22:30.143Z
60d045c4778bafd0fbcfa3f5
/avatars/0cc0c2739c1934430ea09df7e9668c80.svg
Yi Chen
user
ChenYi99
67eca2b8351721d62aa537e0
Yuying Ge
admin_assigned
2025-04-02T09:40:05.288Z
6455cc8f654d8bccae50e4d4
/avatars/506a9992e5bf52e06d37cc22e4b307c0.svg
Yuying Ge
user
tttoaster
67eca2b8351721d62aa537e1
Rui Wang
admin_assigned
2025-04-02T09:40:12.320Z
62e0f1314db2175cd270ad08
/avatars/1d3d6af6c63557f4abf0484e028fa942.svg
Rui Wang
user
ruiwang
67eca2b8351721d62aa537e2
Yixiao Ge
admin_assigned
2025-04-02T09:40:19.876Z
640e9762b03f4cd29f58d982
/avatars/81da37d628163fe3e094b247c7c3a3b5.svg
Yixiao Ge
user
yxgeee
67eca2b8351721d62aa537e3
Lu Qiu
claimed_verified
2025-06-04T09:05:03.244Z
672a037c19f1f942483f680c
/avatars/a48464044e9eb11a2bc062be05d9aa9a.svg
qiulu
user
qiulu66
67eca2b8351721d62aa537e4
Ying Shan
admin_assigned
2025-04-02T09:40:26.551Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
67eca2b8351721d62aa537e5
Xihui Liu
admin_assigned
2025-04-02T09:40:32.831Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
Exploring the Effect of Reinforcement Learning on Video Understanding:
  Insights from SEED-Bench-R1
Recent advancements in Chain of Thought (COT) generation have significantly
improved the reasoning capabilities of Large Language Models (LLMs), with
reinforcement learning (RL) emerging as an effective post-training approach.
Multimodal Large Language Models (MLLMs) inherit this reasoning potential but
remain underexplored in tasks requiring both perception and logical reasoning.
To address this, we introduce SEED-Bench-R1, a benchmark designed to
systematically evaluate post-training methods for MLLMs in video understanding.
It includes intricate real-world videos and complex everyday planning tasks in
the format of multiple-choice questions, requiring sophisticated perception and
reasoning. SEED-Bench-R1 assesses generalization through a three-level
hierarchy: in-distribution, cross-environment, and cross-environment-task
scenarios, equipped with a large-scale training dataset with easily verifiable
ground-truth answers. Using Qwen2-VL-Instruct-7B as a base model, we compare RL
with supervised fine-tuning (SFT), demonstrating RL's data efficiency and
superior performance on both in-distribution and out-of-distribution tasks,
even outperforming SFT on general video understanding benchmarks like
LongVideoBench. Our detailed analysis reveals that RL enhances visual
perception but often produces less logically coherent reasoning chains. We
identify key limitations such as inconsistent reasoning and overlooked visual
cues, and suggest future improvements in base model reasoning, reward modeling,
and RL robustness against noisy signals.
A benchmark (SEED-Bench-R1) evaluates reinforcement learning versus supervised fine-tuning for multimodal large language models in video understanding, highlighting RL's data efficiency and superior performance but identifying limitations in logical coherence and visual cue processing.
Chain of Thought (COT) Large Language Models (LLMs) reinforcement learning (RL) Multimodal Large Language Models (MLLMs) video understanding multiple-choice questions Qwen2-VL-Instruct-7B LongVideoBench supervised fine-tuning (SFT) visual perception reasoning chains reward modeling RL robustness