RynnEC: Bringing MLLMs into Embodied World
We introduce RynnEC, a video multimodal large language model designed for
embodied cognition. Built upon a general-purpose vision-language foundation
model, RynnEC incorporates a region encoder and a mask decoder, enabling
flexible region-level video interaction. Despite its compact architecture,
RynnEC achieves state-of-the-art performance in object property understanding,
object segmentation, and spatial reasoning. Conceptually, it offers a
region-centric video paradigm for the brain of embodied agents, providing
fine-grained perception of the physical world and enabling more precise
interactions. To mitigate the scarcity of annotated 3D datasets, we propose an
egocentric video based pipeline for generating embodied cognition data.
Furthermore, we introduce RynnEC-Bench, a region-centered benchmark for
evaluating embodied cognitive capabilities. We anticipate that RynnEC will
advance the development of general-purpose cognitive cores for embodied agents
and facilitate generalization across diverse embodied tasks. The code, model
checkpoints, and benchmark are available at:
https://github.com/alibaba-damo-academy/RynnEC
68a68f329e4b49496aac697f
Ronghao Dang
admin_assigned
2025-08-21T10:42:56.047Z
64731a68a7f23affe7736d3d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8wESKLFcS2ltPzL-wpG4Z.jpeg
Ronghao Dang
user
RH-Dang
68a68f329e4b49496aac6980
Yuqian Yuan
admin_assigned
2025-08-21T10:42:42.962Z
64a3fe3dde901eb01df12398
https://cdn-avatars.huggingface.co/v1/production/uploads/64a3fe3dde901eb01df12398/Js2bEx4rxKuEKVt5z9I2D.jpeg
YuqianYuan
user
CircleRadon
68a68f329e4b49496aac6981
Yunxuan Mao
admin_assigned
2025-08-21T10:43:01.835Z
67fcc97cede5c434e0cc37e3
/avatars/b07e0a4744c1045828a621146ee6d3c2.svg
yunxuan mao
user
maoyunxuan
68a68f329e4b49496aac6982
Kehan Li
admin_assigned
2025-08-21T10:43:08.690Z
66836aba7b50b433cd681182
/avatars/483a6fef6d27680d19f9ea1789122bdd.svg
Kehan LI
user
CausalLi
68a68f329e4b49496aac6983
Jiangpin Liu
admin_assigned
2025-08-21T10:43:13.996Z
67a6082b2f32323bfb5e6641
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/NZvjm4Kapc7AgReexgRgi.png
jiangpin
user
jiangpinliu
68a68f329e4b49496aac6984
Zhikai Wang
68a68f329e4b49496aac6985
Xin Li
68a68f329e4b49496aac6986
Fan Wang
68a68f329e4b49496aac6987
Deli Zhao
RynnEC: Bringing MLLMs into Embodied World
We introduce RynnEC, a video multimodal large language model designed for
embodied cognition. Built upon a general-purpose vision-language foundation
model, RynnEC incorporates a region encoder and a mask decoder, enabling
flexible region-level video interaction. Despite its compact architecture,
RynnEC achieves state-of-the-art performance in object property understanding,
object segmentation, and spatial reasoning. Conceptually, it offers a
region-centric video paradigm for the brain of embodied agents, providing
fine-grained perception of the physical world and enabling more precise
interactions. To mitigate the scarcity of annotated 3D datasets, we propose an
egocentric video based pipeline for generating embodied cognition data.
Furthermore, we introduce RynnEC-Bench, a region-centered benchmark for
evaluating embodied cognitive capabilities. We anticipate that RynnEC will
advance the development of general-purpose cognitive cores for embodied agents
and facilitate generalization across diverse embodied tasks. The code, model
checkpoints, and benchmark are available at:
https://github.com/alibaba-damo-academy/RynnEC
RynnEC, a video multimodal large language model with a region-centric approach, achieves state-of-the-art performance in object property understanding, segmentation, and spatial reasoning, using an egocentric video pipeline and a region-centered benchmark.
video multimodal large language model embodied cognition region encoder mask decoder region-level video interaction object property understanding object segmentation spatial reasoning egocentric video RynnEC-Bench embodied cognitive capabilities