F1: A Vision-Language-Action Model Bridging Understanding and Generation
  to Actions
Executing language-conditioned tasks in dynamic visual environments remains a
central challenge in embodied AI. Existing Vision-Language-Action (VLA) models
predominantly adopt reactive state-to-action mappings, often leading to
short-sighted behaviors and poor robustness in dynamic scenes. In this paper,
we introduce F1, a pretrained VLA framework which integrates the visual
foresight generation into decision-making pipeline. F1 adopts a
Mixture-of-Transformer architecture with dedicated modules for perception,
foresight generation, and control, thereby bridging understanding, generation,
and actions. At its core, F1 employs a next-scale prediction mechanism to
synthesize goal-conditioned visual foresight as explicit planning targets. By
forecasting plausible future visual states, F1 reformulates action generation
as a foresight-guided inverse dynamics problem, enabling actions that
implicitly achieve visual goals. To endow F1 with robust and generalizable
capabilities, we propose a three-stage training recipe on an extensive dataset
comprising over 330k trajectories across 136 diverse tasks. This training
scheme enhances modular reasoning and equips the model with transferable visual
foresight, which is critical for complex and dynamic environments. Extensive
evaluations on real-world tasks and simulation benchmarks demonstrate F1
consistently outperforms existing approaches, achieving substantial gains in
both task success rate and generalization ability.
68bfa3f2207285de11b07c07
Qi Lv
claimed_verified
2025-09-09T13:47:30.614Z
678123194248fde89e4fc9bf
/avatars/c207e25e12a138eed048c72b143b9bcf.svg
aopolin-lv
user
aopolin-lv
68bfa3f2207285de11b07c08
Weijie Kong
68bfa3f2207285de11b07c09
Hao Li
68bfa3f2207285de11b07c0a
Jia Zeng
68bfa3f2207285de11b07c0b
Zherui Qiu
68bfa3f2207285de11b07c0c
Delin Qu
claimed_verified
2025-09-12T16:20:03.534Z
64daecec888b7e9c400f59b5
https://cdn-avatars.huggingface.co/v1/production/uploads/64daecec888b7e9c400f59b5/f4pfOfWk6jYJX-Nf2-qHn.png
Delin Qu
user
delinqu
68bfa3f2207285de11b07c0d
Haoming Song
68bfa3f2207285de11b07c0e
Qizhi Chen
68bfa3f2207285de11b07c0f
Xiang Deng
68bfa3f2207285de11b07c10
Jiangmiao Pang
F1: A Vision-Language-Action Model Bridging Understanding and Generation
  to Actions
Executing language-conditioned tasks in dynamic visual environments remains a
central challenge in embodied AI. Existing Vision-Language-Action (VLA) models
predominantly adopt reactive state-to-action mappings, often leading to
short-sighted behaviors and poor robustness in dynamic scenes. In this paper,
we introduce F1, a pretrained VLA framework which integrates the visual
foresight generation into decision-making pipeline. F1 adopts a
Mixture-of-Transformer architecture with dedicated modules for perception,
foresight generation, and control, thereby bridging understanding, generation,
and actions. At its core, F1 employs a next-scale prediction mechanism to
synthesize goal-conditioned visual foresight as explicit planning targets. By
forecasting plausible future visual states, F1 reformulates action generation
as a foresight-guided inverse dynamics problem, enabling actions that
implicitly achieve visual goals. To endow F1 with robust and generalizable
capabilities, we propose a three-stage training recipe on an extensive dataset
comprising over 330k trajectories across 136 diverse tasks. This training
scheme enhances modular reasoning and equips the model with transferable visual
foresight, which is critical for complex and dynamic environments. Extensive
evaluations on real-world tasks and simulation benchmarks demonstrate F1
consistently outperforms existing approaches, achieving substantial gains in
both task success rate and generalization ability.
F1, a pretrained VLA framework with foresight generation, improves task success and generalization in dynamic environments through a Mixture-of-Transformer architecture and next-scale prediction.
Mixture-of-Transformer foresight generation next-scale prediction inverse dynamics problem modular reasoning transferable visual foresight