CyberV: Cybernetics for Test-time Scaling in Video Understanding
Current Multimodal Large Language Models (MLLMs) may struggle with
understanding long or complex videos due to computational demands at test time,
lack of robustness, and limited accuracy, primarily stemming from their
feed-forward processing nature. These limitations could be more severe for
models with fewer parameters. To address these limitations, we propose a novel
framework inspired by cybernetic principles, redesigning video MLLMs as
adaptive systems capable of self-monitoring, self-correction, and dynamic
resource allocation during inference. Our approach, CyberV, introduces a
cybernetic loop consisting of an MLLM Inference System, a Sensor, and a
Controller. Specifically, the sensor monitors forward processes of the MLLM and
collects intermediate interpretations, such as attention drift, then the
controller determines when and how to trigger self-correction and generate
feedback to guide the next round. This test-time adaptive scaling framework
enhances frozen MLLMs without requiring retraining or additional components.
Experiments demonstrate significant improvements: CyberV boosts Qwen2.5-VL-7B
by 8.3% and InternVL3-8B by 5.5% on VideoMMMU, surpassing the competitive
proprietary model GPT-4o. When applied to Qwen2.5-VL-72B, it yields a 10.0%
improvement, achieving performance even comparable to human experts.
Furthermore, our method demonstrates consistent gains on general-purpose
benchmarks, such as VideoMME and WorldSense, highlighting its effectiveness and
generalization capabilities in making MLLMs more robust and accurate for
dynamic video understanding. The code is released at
https://github.com/marinero4972/CyberV.
684857ee4fe3b60e21b2589f
Jiahao Meng
claimed_verified
2025-06-11T08:35:52.772Z
65a28e129acab19980226731
/avatars/abc3828f807efc4e03837b0eae063f98.svg
Jiahao Meng
user
marinero4972
684857ee4fe3b60e21b258a0
Shuyang Sun
684857ee4fe3b60e21b258a1
Yue Tan
684857ee4fe3b60e21b258a2
Lu Qi
684857ee4fe3b60e21b258a3
Yunhai Tong
684857ee4fe3b60e21b258a4
Xiangtai Li
extracted_pending
2025-06-10T16:06:07.068Z
6773a00964e8b7438ae809a0
/avatars/83e44d89eefdacbec2c0176e9756977b.svg
Li
user
Xiangtai
684857ee4fe3b60e21b258a5
Longyin Wen
claimed_verified
2025-06-11T08:35:48.618Z
63958b4414513eaf9029ebf1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U1g5H071pWRswGAG9UTpo.png
Xiangtai Li
user
LXT
CyberV: Cybernetics for Test-time Scaling in Video Understanding
Current Multimodal Large Language Models (MLLMs) may struggle with
understanding long or complex videos due to computational demands at test time,
lack of robustness, and limited accuracy, primarily stemming from their
feed-forward processing nature. These limitations could be more severe for
models with fewer parameters. To address these limitations, we propose a novel
framework inspired by cybernetic principles, redesigning video MLLMs as
adaptive systems capable of self-monitoring, self-correction, and dynamic
resource allocation during inference. Our approach, CyberV, introduces a
cybernetic loop consisting of an MLLM Inference System, a Sensor, and a
Controller. Specifically, the sensor monitors forward processes of the MLLM and
collects intermediate interpretations, such as attention drift, then the
controller determines when and how to trigger self-correction and generate
feedback to guide the next round. This test-time adaptive scaling framework
enhances frozen MLLMs without requiring retraining or additional components.
Experiments demonstrate significant improvements: CyberV boosts Qwen2.5-VL-7B
by 8.3% and InternVL3-8B by 5.5% on VideoMMMU, surpassing the competitive
proprietary model GPT-4o. When applied to Qwen2.5-VL-72B, it yields a 10.0%
improvement, achieving performance even comparable to human experts.
Furthermore, our method demonstrates consistent gains on general-purpose
benchmarks, such as VideoMME and WorldSense, highlighting its effectiveness and
generalization capabilities in making MLLMs more robust and accurate for
dynamic video understanding. The code is released at
https://github.com/marinero4972/CyberV.
CyberV enhances Multimodal Large Language Models for video understanding through an adaptive, self-monitoring, and feedback-driven framework, improving performance on various benchmarks.
Multimodal Large Language Models MLLMs feed-forward processing cybernetic principles adaptive systems self-monitoring self-correction dynamic resource allocation cybernetic loop MLLM Inference System Sensor Controller attention drift test-time adaptive scaling VideoMMMU Qwen2.5-VL-7B InternVL3-8B GPT-4o VideoMME WorldSense