See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned
  Aerial Navigation
We present See, Point, Fly (SPF), a training-free aerial vision-and-language
navigation (AVLN) framework built atop vision-language models (VLMs). SPF is
capable of navigating to any goal based on any type of free-form instructions
in any kind of environment. In contrast to existing VLM-based approaches that
treat action prediction as a text generation task, our key insight is to
consider action prediction for AVLN as a 2D spatial grounding task. SPF
harnesses VLMs to decompose vague language instructions into iterative
annotation of 2D waypoints on the input image. Along with the predicted
traveling distance, SPF transforms predicted 2D waypoints into 3D displacement
vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the
traveling distance to facilitate more efficient navigation. Notably, SPF
performs navigation in a closed-loop control manner, enabling UAVs to follow
dynamic targets in dynamic environments. SPF sets a new state of the art in DRL
simulation benchmark, outperforming the previous best method by an absolute
margin of 63%. In extensive real-world evaluations, SPF outperforms strong
baselines by a large margin. We also conduct comprehensive ablation studies to
highlight the effectiveness of our design choice. Lastly, SPF shows remarkable
generalization to different VLMs. Project page: https://spf-web.pages.dev
68d9f3590177a6054b013b0f
Chih Yao Hu
68d9f3590177a6054b013b10
Yang-Sen Lin
claimed_verified
2025-10-06T12:49:21.051Z
671a2e25eaeb8a2c942bc142
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/tjphhS7h_Bh90xF-Et3iN.png
Liam Lin
user
Liammm07
68d9f3590177a6054b013b11
Yuna Lee
claimed_verified
2025-09-29T05:58:13.657Z
631c073e9edb6d320a175b50
https://cdn-avatars.huggingface.co/v1/production/uploads/1662781228166-noauth.png
yuna0x0
user
yuna0x0
68d9f3590177a6054b013b12
Chih-Hai Su
68d9f3590177a6054b013b13
Jie-Ying Lee
claimed_verified
2025-09-29T05:58:09.121Z
655f1770f74fa124d1172ec1
https://cdn-avatars.huggingface.co/v1/production/uploads/655f1770f74fa124d1172ec1/bdYocZ1qN50CAfb2z2YLA.png
Jie-Ying Lee
user
jayinnn
68d9f3590177a6054b013b14
Shr-Ruei Tsai
68d9f3590177a6054b013b15
Chin-Yang Lin
68d9f3590177a6054b013b16
Kuan-Wen Chen
68d9f3590177a6054b013b17
Tsung-Wei Ke
68d9f3590177a6054b013b18
Yu-Lun Liu
See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned
  Aerial Navigation
We present See, Point, Fly (SPF), a training-free aerial vision-and-language
navigation (AVLN) framework built atop vision-language models (VLMs). SPF is
capable of navigating to any goal based on any type of free-form instructions
in any kind of environment. In contrast to existing VLM-based approaches that
treat action prediction as a text generation task, our key insight is to
consider action prediction for AVLN as a 2D spatial grounding task. SPF
harnesses VLMs to decompose vague language instructions into iterative
annotation of 2D waypoints on the input image. Along with the predicted
traveling distance, SPF transforms predicted 2D waypoints into 3D displacement
vectors as action commands for UAVs. Moreover, SPF also adaptively adjusts the
traveling distance to facilitate more efficient navigation. Notably, SPF
performs navigation in a closed-loop control manner, enabling UAVs to follow
dynamic targets in dynamic environments. SPF sets a new state of the art in DRL
simulation benchmark, outperforming the previous best method by an absolute
margin of 63%. In extensive real-world evaluations, SPF outperforms strong
baselines by a large margin. We also conduct comprehensive ablation studies to
highlight the effectiveness of our design choice. Lastly, SPF shows remarkable
generalization to different VLMs. Project page: https://spf-web.pages.dev
See, Point, Fly (SPF) is a training-free aerial vision-and-language navigation framework that treats action prediction as a 2D spatial grounding task, outperforming existing methods in both simulation and real-world evaluations.
vision-language models VLMs action prediction 2D spatial grounding 2D waypoints 3D displacement vectors UAVs closed-loop control DRL simulation benchmark ablation studies