DiffSemanticFusion: Semantic Raster BEV Fusion for Autonomous Driving
  via Online HD Map Diffusion
Autonomous driving requires accurate scene understanding, including road
geometry, traffic agents, and their semantic relationships. In online HD map
generation scenarios, raster-based representations are well-suited to vision
models but lack geometric precision, while graph-based representations retain
structural detail but become unstable without precise maps. To harness the
complementary strengths of both, we propose DiffSemanticFusion -- a fusion
framework for multimodal trajectory prediction and planning. Our approach
reasons over a semantic raster-fused BEV space, enhanced by a map diffusion
module that improves both the stability and expressiveness of online HD map
representations. We validate our framework on two downstream tasks: trajectory
prediction and planning-oriented end-to-end autonomous driving. Experiments on
real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate
improved performance over several state-of-the-art methods. For the prediction
task on nuScenes, we integrate DiffSemanticFusion with the online HD map
informed QCNet, achieving a 5.1\% performance improvement. For end-to-end
autonomous driving in NAVSIM, DiffSemanticFusion achieves state-of-the-art
results, with a 15\% performance gain in NavHard scenarios. In addition,
extensive ablation and sensitivity studies show that our map diffusion module
can be seamlessly integrated into other vector-based approaches to enhance
performance. All artifacts are available at
https://github.com/SunZhigang7/DiffSemanticFusion.
68943a7a741a16f544fbcf96
Zhigang Sun
claimed_verified
2025-08-07T10:37:52.546Z
6819f16fa41ec6a2ca4cb88d
/avatars/8ba11096e942aafd32025fe0655a2bae.svg
SunZhigang
user
SunZhigang7
68943a7a741a16f544fbcf97
Yiru Wang
68943a7a741a16f544fbcf98
Anqing Jiang
68943a7a741a16f544fbcf99
Shuo Wang
68943a7a741a16f544fbcf9a
Yu Gao
68943a7a741a16f544fbcf9b
Yuwen Heng
68943a7a741a16f544fbcf9c
Shouyi Zhang
68943a7a741a16f544fbcf9d
An He
68943a7a741a16f544fbcf9e
Hao Jiang
68943a7a741a16f544fbcf9f
Jinhao Chai
68943a7a741a16f544fbcfa0
Zichong Gu
68943a7a741a16f544fbcfa1
Wang Jijun
68943a7a741a16f544fbcfa2
Shichen Tang
68943a7a741a16f544fbcfa3
Lavdim Halilaj
68943a7a741a16f544fbcfa4
Juergen Luettin
68943a7a741a16f544fbcfa5
Hao Sun
DiffSemanticFusion: Semantic Raster BEV Fusion for Autonomous Driving
  via Online HD Map Diffusion
Autonomous driving requires accurate scene understanding, including road
geometry, traffic agents, and their semantic relationships. In online HD map
generation scenarios, raster-based representations are well-suited to vision
models but lack geometric precision, while graph-based representations retain
structural detail but become unstable without precise maps. To harness the
complementary strengths of both, we propose DiffSemanticFusion -- a fusion
framework for multimodal trajectory prediction and planning. Our approach
reasons over a semantic raster-fused BEV space, enhanced by a map diffusion
module that improves both the stability and expressiveness of online HD map
representations. We validate our framework on two downstream tasks: trajectory
prediction and planning-oriented end-to-end autonomous driving. Experiments on
real-world autonomous driving benchmarks, nuScenes and NAVSIM, demonstrate
improved performance over several state-of-the-art methods. For the prediction
task on nuScenes, we integrate DiffSemanticFusion with the online HD map
informed QCNet, achieving a 5.1\% performance improvement. For end-to-end
autonomous driving in NAVSIM, DiffSemanticFusion achieves state-of-the-art
results, with a 15\% performance gain in NavHard scenarios. In addition,
extensive ablation and sensitivity studies show that our map diffusion module
can be seamlessly integrated into other vector-based approaches to enhance
performance. All artifacts are available at
https://github.com/SunZhigang7/DiffSemanticFusion.
DiffSemanticFusion enhances autonomous driving by fusing semantic raster and graph-based representations using a map diffusion module, improving trajectory prediction and end-to-end driving performance.
DiffSemanticFusion semantic raster-fused BEV space map diffusion module trajectory prediction end-to-end autonomous driving nuScenes NAVSIM NavHard scenarios