Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal
  Learning
Modern robot navigation systems encounter difficulties in diverse and complex
indoor environments. Traditional approaches rely on multiple modules with small
models or rule-based systems and thus lack adaptability to new environments. To
address this, we developed Astra, a comprehensive dual-model architecture,
Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a
multimodal LLM, processes vision and language inputs to perform self and goal
localization using a hybrid topological-semantic graph as the global map, and
outperforms traditional visual place recognition methods. Astra-Local, a
multitask network, handles local path planning and odometry estimation. Its 4D
spatial-temporal encoder, trained through self-supervised learning, generates
robust 4D features for downstream tasks. The planning head utilizes flow
matching and a novel masked ESDF loss to minimize collision risks for
generating local trajectories, and the odometry head integrates multi-sensor
inputs via a transformer encoder to predict the relative pose of the robot.
Deployed on real in-house mobile robots, Astra achieves high end-to-end mission
success rate across diverse indoor environments.
6846ca9b3ec10bdd8ab4dbf4
Sheng Chen
claimed_verified
2025-06-10T09:21:08.452Z
66727b038171db46e7f4f242
/avatars/3f8ab656006533fe1a24dd48c9b0a1b6.svg
sc
user
sc-bd
6846ca9b3ec10bdd8ab4dbf5
Peiyu He
6846ca9b3ec10bdd8ab4dbf6
Jiaxin Hu
6846ca9b3ec10bdd8ab4dbf7
Ziyang Liu
6846ca9b3ec10bdd8ab4dbf8
Yansheng Wang
6846ca9b3ec10bdd8ab4dbf9
Tao Xu
6846ca9b3ec10bdd8ab4dbfa
Chi Zhang
6846ca9b3ec10bdd8ab4dbfb
Chongchong Zhang
6846ca9b3ec10bdd8ab4dbfc
Chao An
6846ca9b3ec10bdd8ab4dbfd
Shiyu Cai
6846ca9b3ec10bdd8ab4dbfe
Duo Cao
6846ca9b3ec10bdd8ab4dbff
Kangping Chen
6846ca9b3ec10bdd8ab4dc00
Shuai Chu
6846ca9b3ec10bdd8ab4dc01
Tianwei Chu
6846ca9b3ec10bdd8ab4dc02
Mingdi Dan
6846ca9b3ec10bdd8ab4dc03
Min Du
6846ca9b3ec10bdd8ab4dc04
Weiwei Fang
6846ca9b3ec10bdd8ab4dc05
Pengyou Fu
6846ca9b3ec10bdd8ab4dc06
Junkai Hu
6846ca9b3ec10bdd8ab4dc07
Xiaowei Jiang
6846ca9b3ec10bdd8ab4dc08
Zhaodi Jiang
6846ca9b3ec10bdd8ab4dc09
Fuxuan Li
6846ca9b3ec10bdd8ab4dc0a
Jun Li
6846ca9b3ec10bdd8ab4dc0b
Minghui Li
6846ca9b3ec10bdd8ab4dc0c
Mingyao Li
6846ca9b3ec10bdd8ab4dc0d
Yanchang Li
6846ca9b3ec10bdd8ab4dc0e
Zhibin Li
6846ca9b3ec10bdd8ab4dc0f
Guangming Liu
6846ca9b3ec10bdd8ab4dc10
Kairui Liu
6846ca9b3ec10bdd8ab4dc11
Lihao Liu
6846ca9b3ec10bdd8ab4dc12
Weizhi Liu
6846ca9b3ec10bdd8ab4dc13
Xiaoshun Liu
6846ca9b3ec10bdd8ab4dc14
Yufei Liu
6846ca9b3ec10bdd8ab4dc15
Yunfei Liu
6846ca9b3ec10bdd8ab4dc16
Qiang Lu
6846ca9b3ec10bdd8ab4dc17
Yuanfei Luo
6846ca9b3ec10bdd8ab4dc18
Xiang Lv
6846ca9b3ec10bdd8ab4dc19
Hongying Ma
6846ca9b3ec10bdd8ab4dc1a
Sai Ma
6846ca9b3ec10bdd8ab4dc1b
Lingxian Mi
6846ca9b3ec10bdd8ab4dc1c
Sha Sa
6846ca9b3ec10bdd8ab4dc1d
Hongxiang Shu
6846ca9b3ec10bdd8ab4dc1e
Lei Tian
6846ca9b3ec10bdd8ab4dc1f
Chengzhi Wang
6846ca9b3ec10bdd8ab4dc20
Jiayu Wang
6846ca9b3ec10bdd8ab4dc21
Kaijie Wang
6846ca9b3ec10bdd8ab4dc22
Qingyi Wang
6846ca9b3ec10bdd8ab4dc23
Renwen Wang
6846ca9b3ec10bdd8ab4dc24
Tao Wang
6846ca9b3ec10bdd8ab4dc25
Wei Wang
6846ca9b3ec10bdd8ab4dc26
Xirui Wang
6846ca9b3ec10bdd8ab4dc27
Chao Wei
6846ca9b3ec10bdd8ab4dc28
Xuguang Wei
6846ca9b3ec10bdd8ab4dc29
Zijun Xia
6846ca9b3ec10bdd8ab4dc2a
Zhaohao Xiao
6846ca9b3ec10bdd8ab4dc2b
Tingshuai Yan
6846ca9b3ec10bdd8ab4dc2c
Liyan Yang
6846ca9b3ec10bdd8ab4dc2d
Yifan Yang
6846ca9b3ec10bdd8ab4dc2e
Zhikai Yang
6846ca9b3ec10bdd8ab4dc2f
Zhong Yin
6846ca9b3ec10bdd8ab4dc30
Li Yuan
6846ca9b3ec10bdd8ab4dc31
Liuchun Yuan
6846ca9b3ec10bdd8ab4dc32
Chi Zhang
6846ca9b3ec10bdd8ab4dc33
Jinyang Zhang
6846ca9b3ec10bdd8ab4dc34
Junhui Zhang
6846ca9b3ec10bdd8ab4dc35
Linge Zhang
6846ca9b3ec10bdd8ab4dc36
Zhenyi Zhang
6846ca9b3ec10bdd8ab4dc37
Zheyu Zhang
6846ca9b3ec10bdd8ab4dc38
Dongjie Zhu
6846ca9b3ec10bdd8ab4dc39
Hang Li
6846ca9b3ec10bdd8ab4dc3a
Yangang Zhang
Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal
  Learning
Modern robot navigation systems encounter difficulties in diverse and complex
indoor environments. Traditional approaches rely on multiple modules with small
models or rule-based systems and thus lack adaptability to new environments. To
address this, we developed Astra, a comprehensive dual-model architecture,
Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a
multimodal LLM, processes vision and language inputs to perform self and goal
localization using a hybrid topological-semantic graph as the global map, and
outperforms traditional visual place recognition methods. Astra-Local, a
multitask network, handles local path planning and odometry estimation. Its 4D
spatial-temporal encoder, trained through self-supervised learning, generates
robust 4D features for downstream tasks. The planning head utilizes flow
matching and a novel masked ESDF loss to minimize collision risks for
generating local trajectories, and the odometry head integrates multi-sensor
inputs via a transformer encoder to predict the relative pose of the robot.
Deployed on real in-house mobile robots, Astra achieves high end-to-end mission
success rate across diverse indoor environments.
Astra, a dual-model architecture for mobile robot navigation, uses a multimodal LLM for global localization and a multitask network for local path planning and odometry estimation, achieving high success rates in diverse indoor environments.
LLM self and goal localization hybrid topological-semantic graph multimodal LLM multitask network 4D spatial-temporal encoder self-supervised learning 4D features flow matching masked ESDF loss local trajectories transformer encoder relative pose prediction