StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context
  Modeling
Vision-and-Language Navigation (VLN) in real-world settings requires agents
to process continuous visual streams and generate actions with low latency
grounded in language instructions. While Video-based Large Language Models
(Video-LLMs) have driven recent progress, current VLN methods based on
Video-LLM often face trade-offs among fine-grained visual understanding,
long-term context modeling and computational efficiency. We introduce
StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context
modeling strategy to support multi-modal reasoning over interleaved vision,
language and action inputs. The fast-streaming dialogue context facilitates
responsive action generation through a sliding-window of active dialogues,
while the slow-updating memory context compresses historical visual states
using a 3D-aware token pruning strategy. With this slow-fast design, StreamVLN
achieves coherent multi-turn dialogue through efficient KV cache reuse,
supporting long video streams with bounded context size and inference cost.
Experiments on VLN-CE benchmarks demonstrate state-of-the-art performance with
stable low latency, ensuring robustness and efficiency in real-world
deployment. The project page is:
https://streamvln.github.io/{https://streamvln.github.io/}.
686de9d4cb5725779c60b487
Meng Wei
686de9d4cb5725779c60b488
Chenyang Wan
claimed_verified
2025-07-09T08:49:45.781Z
66bb5e6573ce3e3ef046615a
/avatars/cbfb4b4114dc3afd0eb63b43a809ba09.svg
Chenyang Wan
user
cywan
686de9d4cb5725779c60b489
Xiqian Yu
686de9d4cb5725779c60b48a
Tai Wang
686de9d4cb5725779c60b48b
Yuqiang Yang
686de9d4cb5725779c60b48c
Xiaohan Mao
686de9d4cb5725779c60b48d
Chenming Zhu
686de9d4cb5725779c60b48e
Wenzhe Cai
686de9d4cb5725779c60b48f
Hanqing Wang
686de9d4cb5725779c60b490
Yilun Chen
686de9d4cb5725779c60b491
Xihui Liu
686de9d4cb5725779c60b492
Jiangmiao Pang
StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context
  Modeling
Vision-and-Language Navigation (VLN) in real-world settings requires agents
to process continuous visual streams and generate actions with low latency
grounded in language instructions. While Video-based Large Language Models
(Video-LLMs) have driven recent progress, current VLN methods based on
Video-LLM often face trade-offs among fine-grained visual understanding,
long-term context modeling and computational efficiency. We introduce
StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context
modeling strategy to support multi-modal reasoning over interleaved vision,
language and action inputs. The fast-streaming dialogue context facilitates
responsive action generation through a sliding-window of active dialogues,
while the slow-updating memory context compresses historical visual states
using a 3D-aware token pruning strategy. With this slow-fast design, StreamVLN
achieves coherent multi-turn dialogue through efficient KV cache reuse,
supporting long video streams with bounded context size and inference cost.
Experiments on VLN-CE benchmarks demonstrate state-of-the-art performance with
stable low latency, ensuring robustness and efficiency in real-world
deployment. The project page is:
https://streamvln.github.io/{https://streamvln.github.io/}.
StreamVLN, a streaming VLN framework, uses a hybrid slow-fast context modeling strategy to balance fine-grained visual understanding, long-term context modeling, and computational efficiency in real-world settings.
Video-LLMs StreamVLN hybrid slow-fast context modeling multi-modal reasoning fast-streaming dialogue context slow-updating memory context 3D-aware token pruning KV cache reuse VLN-CE benchmarks