TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning
  for Video LLMs
This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework
designed to improve the effectiveness of adapting multimodal large language
models (MLLMs) to video temporal grounding tasks. We reveal that existing
reinforcement learning methods, such as Group Relative Policy Optimization
(GRPO), rely on on-policy sampling for policy updates. However, in tasks with
large temporal search spaces, this strategy becomes both inefficient and
limited in performance, as it often fails to identify temporally accurate
solutions. To address this limitation, TempSamp-R1 leverages ground-truth
annotations as off-policy supervision to provide temporally precise guidance,
effectively compensating for the sparsity and misalignment in on-policy
solutions. To further stabilize training and reduce variance in reward-based
updates, TempSamp-R1 provides a non-linear soft advantage computation method
that dynamically reshapes the reward feedback via an asymmetric transformation.
By employing a hybrid Chain-of-Thought (CoT) training paradigm, TempSamp-R1
optimizes a single unified model to support both CoT and non-CoT inference
modes, enabling efficient handling of queries with varying reasoning
complexity. Experimental results demonstrate that TempSamp-R1 outperforms
GRPO-based baselines, establishing new state-of-the-art performance on
benchmark datasets: Charades-STA (R1@0.7: 52.9%, +2.7%), ActivityNet Captions
(R1@0.5: 56.0%, +5.3%), and QVHighlights (mAP: 30.0%, +3.0%). Moreover,
TempSamp-R1 shows robust few-shot generalization capabilities under limited
data. Code: https://github.com/HVision-NKU/TempSamp-R1
68d23a2c1ca7156988a8ee4a
Yunheng Li
claimed_verified
2025-09-23T10:05:02.046Z
67485bfd768f8d6a509d5cd7
/avatars/ad01e707a2066ef673ac7317ccdbb902.svg
Yunheng Li
user
lyhisme
68d23a2c1ca7156988a8ee4b
Jing Cheng
68d23a2c1ca7156988a8ee4c
Shaoyong Jia
68d23a2c1ca7156988a8ee4d
Hangyi Kuang
68d23a2c1ca7156988a8ee4e
Shaohui Jiao
68d23a2c1ca7156988a8ee4f
Qibin Hou
68d23a2c1ca7156988a8ee50
Ming-Ming Cheng
TempSamp-R1: Effective Temporal Sampling with Reinforcement Fine-Tuning
  for Video LLMs
This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework
designed to improve the effectiveness of adapting multimodal large language
models (MLLMs) to video temporal grounding tasks. We reveal that existing
reinforcement learning methods, such as Group Relative Policy Optimization
(GRPO), rely on on-policy sampling for policy updates. However, in tasks with
large temporal search spaces, this strategy becomes both inefficient and
limited in performance, as it often fails to identify temporally accurate
solutions. To address this limitation, TempSamp-R1 leverages ground-truth
annotations as off-policy supervision to provide temporally precise guidance,
effectively compensating for the sparsity and misalignment in on-policy
solutions. To further stabilize training and reduce variance in reward-based
updates, TempSamp-R1 provides a non-linear soft advantage computation method
that dynamically reshapes the reward feedback via an asymmetric transformation.
By employing a hybrid Chain-of-Thought (CoT) training paradigm, TempSamp-R1
optimizes a single unified model to support both CoT and non-CoT inference
modes, enabling efficient handling of queries with varying reasoning
complexity. Experimental results demonstrate that TempSamp-R1 outperforms
GRPO-based baselines, establishing new state-of-the-art performance on
benchmark datasets: Charades-STA (R1@0.7: 52.9%, +2.7%), ActivityNet Captions
(R1@0.5: 56.0%, +5.3%), and QVHighlights (mAP: 30.0%, +3.0%). Moreover,
TempSamp-R1 shows robust few-shot generalization capabilities under limited
data. Code: https://github.com/HVision-NKU/TempSamp-R1
TempSamp-R1, a reinforcement fine-tuning framework, enhances multimodal large language models for video temporal grounding by using off-policy supervision and a hybrid Chain-of-Thought training paradigm, achieving state-of-the-art performance on benchmark datasets.
reinforcement fine-tuning multimodal large language models video temporal grounding Group Relative Policy Optimization on-policy sampling off-policy supervision non-linear soft advantage computation Chain-of-Thought training few-shot generalization