ThinkGrasp: A Vision-Language System for Strategic Part Grasping in
  Clutter
Robotic grasping in cluttered environments remains a significant challenge
due to occlusions and complex object arrangements. We have developed
ThinkGrasp, a plug-and-play vision-language grasping system that makes use of
GPT-4o's advanced contextual reasoning for heavy clutter environment grasping
strategies. ThinkGrasp can effectively identify and generate grasp poses for
target objects, even when they are heavily obstructed or nearly invisible, by
using goal-oriented language to guide the removal of obstructing objects. This
approach progressively uncovers the target object and ultimately grasps it with
a few steps and a high success rate. In both simulated and real experiments,
ThinkGrasp achieved a high success rate and significantly outperformed
state-of-the-art methods in heavily cluttered environments or with diverse
unseen objects, demonstrating strong generalization capabilities.
669843f1615c05521cae4c2f
Yaoyao Qian
admin_assigned
2024-07-18T16:47:31.207Z
65c431a609672feb8cac22e7
https://cdn-avatars.huggingface.co/v1/production/uploads/65c431a609672feb8cac22e7/obXCXUOgEoCuJbGuzYOGA.jpeg
Yaoyao Qian
user
FreaxRuby
669843f1615c05521cae4c30
Xupeng Zhu
669843f1615c05521cae4c31
Ondrej Biza
669843f1615c05521cae4c32
Shuo Jiang
669843f1615c05521cae4c33
Linfeng Zhao
669843f1615c05521cae4c34
Haojie Huang
669843f1615c05521cae4c35
Yu Qi
669843f1615c05521cae4c36
Robert Platt
ThinkGrasp: A Vision-Language System for Strategic Part Grasping in
  Clutter
Robotic grasping in cluttered environments remains a significant challenge
due to occlusions and complex object arrangements. We have developed
ThinkGrasp, a plug-and-play vision-language grasping system that makes use of
GPT-4o's advanced contextual reasoning for heavy clutter environment grasping
strategies. ThinkGrasp can effectively identify and generate grasp poses for
target objects, even when they are heavily obstructed or nearly invisible, by
using goal-oriented language to guide the removal of obstructing objects. This
approach progressively uncovers the target object and ultimately grasps it with
a few steps and a high success rate. In both simulated and real experiments,
ThinkGrasp achieved a high success rate and significantly outperformed
state-of-the-art methods in heavily cluttered environments or with diverse
unseen objects, demonstrating strong generalization capabilities.
ThinkGrasp, a vision-language robotic grasping system using GPT-4o, achieves high success rates in heavily cluttered environments by progressively removing occlusions and identifying grasp poses.
GPT-4o contextual reasoning vision-language grasping system grasp poses goal-oriented language