VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement
  Learning
Coordinating multiple embodied agents in dynamic environments remains a core
challenge in artificial intelligence, requiring both perception-driven
reasoning and scalable cooperation strategies. While recent works have
leveraged large language models (LLMs) for multi-agent planning, a few have
begun to explore vision-language models (VLMs) for visual reasoning. However,
these VLM-based approaches remain limited in their support for diverse
embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical
benchmark tailored for embodied multi-agent cooperation, featuring three
structured levels: agent activation, task planning, and trajectory perception.
VIKI-Bench includes diverse robot embodiments, multi-view visual observations,
and structured supervision signals to evaluate reasoning grounded in visual
inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a
two-stage framework that fine-tunes a pretrained vision-language model (VLM)
using Chain-of-Thought annotated demonstrations, followed by reinforcement
learning under multi-level reward signals. Our extensive experiments show that
VIKI-R significantly outperforms baselines method across all task levels.
Furthermore, we show that reinforcement learning enables the emergence of
compositional cooperation patterns among heterogeneous agents. Together,
VIKI-Bench and VIKI-R offer a unified testbed and method for advancing
multi-agent, visual-driven cooperation in embodied AI systems.
6858c341c0c8e29df8ea3c7f
Li Kang
claimed_verified
2025-06-23T08:14:50.165Z
64eadcb03d76028d805a7818
/avatars/528e4fded4419caf08589b2ed40437bc.svg
Li Kang
user
FACEONG
6858c341c0c8e29df8ea3c80
Xiufeng Song
6858c341c0c8e29df8ea3c81
Heng Zhou
claimed_verified
2025-08-18T07:00:17.954Z
660d17d6c9be0dcd31a30b3d
/avatars/3743fe9b695c488ebe33f0d8fd607a8a.svg
Zhou Heng
user
henggg
6858c341c0c8e29df8ea3c82
Yiran Qin
6858c341c0c8e29df8ea3c83
Jie Yang
6858c341c0c8e29df8ea3c84
Xiaohong Liu
6858c341c0c8e29df8ea3c85
Philip Torr
6858c341c0c8e29df8ea3c86
Lei Bai
6858c341c0c8e29df8ea3c87
Zhenfei Yin
claimed_verified
2025-09-03T08:30:53.405Z
64e314ad24809d7fa0f20fbc
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/bHE0w_hjDFvU-Aul0_E7g.jpeg
Zhenfei Yin
user
JeremyYin
VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement
  Learning
Coordinating multiple embodied agents in dynamic environments remains a core
challenge in artificial intelligence, requiring both perception-driven
reasoning and scalable cooperation strategies. While recent works have
leveraged large language models (LLMs) for multi-agent planning, a few have
begun to explore vision-language models (VLMs) for visual reasoning. However,
these VLM-based approaches remain limited in their support for diverse
embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical
benchmark tailored for embodied multi-agent cooperation, featuring three
structured levels: agent activation, task planning, and trajectory perception.
VIKI-Bench includes diverse robot embodiments, multi-view visual observations,
and structured supervision signals to evaluate reasoning grounded in visual
inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a
two-stage framework that fine-tunes a pretrained vision-language model (VLM)
using Chain-of-Thought annotated demonstrations, followed by reinforcement
learning under multi-level reward signals. Our extensive experiments show that
VIKI-R significantly outperforms baselines method across all task levels.
Furthermore, we show that reinforcement learning enables the emergence of
compositional cooperation patterns among heterogeneous agents. Together,
VIKI-Bench and VIKI-R offer a unified testbed and method for advancing
multi-agent, visual-driven cooperation in embodied AI systems.
VIKI-Bench and VIKI-R provide a benchmark and framework for evaluating and improving visual-driven cooperation among diverse embodied agents using vision-language models and reinforcement learning.
embodied agents VIKI-Bench multi-agent cooperation vision-language models VIKI-R Chain-of-Thought reinforcement learning compositional cooperation multi-level reward signals robot embodiments multi-view visual observations structured supervision signals