VLA^2: Empowering Vision-Language-Action Models with an Agentic
  Framework for Unseen Concept Manipulation
Current vision-language-action (VLA) models, pre-trained on large-scale
robotic data, exhibit strong multi-task capabilities and generalize well to
variations in visual and language instructions for manipulation. However, their
success rate drops significantly when faced with object concepts outside the
training data, such as unseen object descriptions and textures in the dataset.
To address this, we propose a novel agentic framework, VLA^2, which leverages
OpenVLA as the execution backbone and effectively leverages external modules
such as web retrieval and object detection to provide visual and textual
knowledge about target objects to the VLA. This approach mitigates
generalization failure when handling out-of-distribution objects. Based on the
LIBERO simulation environment, we introduced novel objects and object
descriptions to construct a new evaluation benchmark with three difficulty
levels to test the effectiveness of our method. Our framework successfully
outperformed the current state-of-the-art models on our designed hard-level
generalization benchmark. Compared to the standalone OpenVLA baseline, VLA^2
achieves a 44.2% improvement in the success rate in the hard-level benchmark
and an average improvement of 20.2% in all customized environments without any
performance degradation on in-domain tasks. Project website:
https://vla-2.github.io.
68f1d24b6e0bef323a68ff4b
Han Zhao
claimed_verified
2025-10-18T04:49:10.107Z
646dbbc8075bbcc48ddcecbf
https://cdn-avatars.huggingface.co/v1/production/uploads/646dbbc8075bbcc48ddcecbf/V52Em-78O5F3QxRbRwG5O.jpeg
Han Zhao
user
han1997
68f1d24b6e0bef323a68ff4c
Jiaxuan Zhang
68f1d24b6e0bef323a68ff4d
Wenxuan Song
68f1d24b6e0bef323a68ff4e
Pengxiang Ding
68f1d24b6e0bef323a68ff4f
Donglin Wang
VLA^2: Empowering Vision-Language-Action Models with an Agentic
  Framework for Unseen Concept Manipulation
Current vision-language-action (VLA) models, pre-trained on large-scale
robotic data, exhibit strong multi-task capabilities and generalize well to
variations in visual and language instructions for manipulation. However, their
success rate drops significantly when faced with object concepts outside the
training data, such as unseen object descriptions and textures in the dataset.
To address this, we propose a novel agentic framework, VLA^2, which leverages
OpenVLA as the execution backbone and effectively leverages external modules
such as web retrieval and object detection to provide visual and textual
knowledge about target objects to the VLA. This approach mitigates
generalization failure when handling out-of-distribution objects. Based on the
LIBERO simulation environment, we introduced novel objects and object
descriptions to construct a new evaluation benchmark with three difficulty
levels to test the effectiveness of our method. Our framework successfully
outperformed the current state-of-the-art models on our designed hard-level
generalization benchmark. Compared to the standalone OpenVLA baseline, VLA^2
achieves a 44.2% improvement in the success rate in the hard-level benchmark
and an average improvement of 20.2% in all customized environments without any
performance degradation on in-domain tasks. Project website:
https://vla-2.github.io.
A novel agentic framework, VLA^2, enhances vision-language-action models by integrating external modules like web retrieval and object detection, improving generalization to unseen objects and descriptions.
vision-language-action models pre-trained models multi-task capabilities generalization out-of-distribution objects OpenVLA web retrieval object detection LIBERO simulation environment evaluation benchmark success rate in-domain tasks