TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning
Recently, improving the reasoning ability of large multimodal models (LMMs)
through reinforcement learning has made great progress. However, most existing
works are based on highly reasoning-intensive datasets such as mathematics and
code, and researchers generally choose large-scale models as the foundation. We
argue that exploring small-scale models' reasoning capabilities remains
valuable for researchers with limited computational resources. Moreover,
enabling models to explain their reasoning processes on general
question-answering datasets is equally meaningful. Therefore, we present the
small-scale video reasoning model TinyLLaVA-Video-R1. Based on TinyLLaVA-Video,
a traceably trained video understanding model with no more than 4B parameters,
it not only demonstrates significantly improved reasoning and thinking
capabilities after using reinforcement learning on general Video-QA datasets,
but also exhibits the emergent characteristic of "aha moments". Furthermore, we
share a series of experimental findings, aiming to provide practical insights
for future exploration of video reasoning (thinking) abilities in small-scale
models. It is available at https://github.com/ZhangXJ199/TinyLLaVA-Video-R1.
67fdceb2df4261c001394f59
Xingjian Zhang
claimed_verified
2025-04-15T07:54:14.629Z
66448136bfe15e84d3987372
https://cdn-avatars.huggingface.co/v1/production/uploads/66448136bfe15e84d3987372/SFEbWGgqhdZjxCS7qF3YL.jpeg
Zhang Xingjian
user
Zhang199
67fdceb2df4261c001394f5a
Siwei Wen
admin_assigned
2025-04-15T08:29:04.694Z
67f72edbc791c2e0f938203d
/avatars/c8e6d5a2f2122482e5fab7c6438440b5.svg
si wei wen
user
wenzz1
67fdceb2df4261c001394f5b
Wenjun Wu
67fdceb2df4261c001394f5c
Lei Huang
TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning
Recently, improving the reasoning ability of large multimodal models (LMMs)
through reinforcement learning has made great progress. However, most existing
works are based on highly reasoning-intensive datasets such as mathematics and
code, and researchers generally choose large-scale models as the foundation. We
argue that exploring small-scale models' reasoning capabilities remains
valuable for researchers with limited computational resources. Moreover,
enabling models to explain their reasoning processes on general
question-answering datasets is equally meaningful. Therefore, we present the
small-scale video reasoning model TinyLLaVA-Video-R1. Based on TinyLLaVA-Video,
a traceably trained video understanding model with no more than 4B parameters,
it not only demonstrates significantly improved reasoning and thinking
capabilities after using reinforcement learning on general Video-QA datasets,
but also exhibits the emergent characteristic of "aha moments". Furthermore, we
share a series of experimental findings, aiming to provide practical insights
for future exploration of video reasoning (thinking) abilities in small-scale
models. It is available at https://github.com/ZhangXJ199/TinyLLaVA-Video-R1.
TinyLLaVA-Video-R1, a small-scale video reasoning model, demonstrates improved reasoning capabilities and exhibits "aha moments" after reinforcement learning on general Video-QA datasets.
large multimodal models reinforcement learning small-scale models general question-answering datasets video understanding model TinyLLaVA-Video Video-QA datasets reasoning and thinking capabilities aha moments