PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving
While end-to-end autonomous driving models show promising results, their
practical deployment is often hindered by large model sizes, a reliance on
expensive LiDAR sensors and computationally intensive BEV feature
representations. This limits their scalability, especially for mass-market
vehicles equipped only with cameras. To address these challenges, we propose
PRIX (Plan from Raw Pixels). Our novel and efficient end-to-end driving
architecture operates using only camera data, without explicit BEV
representation and forgoing the need for LiDAR. PRIX leverages a visual feature
extractor coupled with a generative planning head to predict safe trajectories
from raw pixel inputs directly. A core component of our architecture is the
Context-aware Recalibration Transformer (CaRT), a novel module designed to
effectively enhance multi-level visual features for more robust planning. We
demonstrate through comprehensive experiments that PRIX achieves
state-of-the-art performance on the NavSim and nuScenes benchmarks, matching
the capabilities of larger, multimodal diffusion planners while being
significantly more efficient in terms of inference speed and model size, making
it a practical solution for real-world deployment. Our work is open-source and
the code will be at https://maxiuw.github.io/prix.
688223086a54dd1e77daa947
Maciej K. Wozniak
claimed_verified
2025-07-24T14:11:19.043Z
641862440956be7233a1788f
/avatars/8710a44806a98f44d36b60814144186a.svg
Maciej Wozniak
user
maciejw94
688223086a54dd1e77daa948
Lianhang Liu
688223086a54dd1e77daa949
Yixi Cai
688223086a54dd1e77daa94a
Patric Jensfelt
PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Driving
While end-to-end autonomous driving models show promising results, their
practical deployment is often hindered by large model sizes, a reliance on
expensive LiDAR sensors and computationally intensive BEV feature
representations. This limits their scalability, especially for mass-market
vehicles equipped only with cameras. To address these challenges, we propose
PRIX (Plan from Raw Pixels). Our novel and efficient end-to-end driving
architecture operates using only camera data, without explicit BEV
representation and forgoing the need for LiDAR. PRIX leverages a visual feature
extractor coupled with a generative planning head to predict safe trajectories
from raw pixel inputs directly. A core component of our architecture is the
Context-aware Recalibration Transformer (CaRT), a novel module designed to
effectively enhance multi-level visual features for more robust planning. We
demonstrate through comprehensive experiments that PRIX achieves
state-of-the-art performance on the NavSim and nuScenes benchmarks, matching
the capabilities of larger, multimodal diffusion planners while being
significantly more efficient in terms of inference speed and model size, making
it a practical solution for real-world deployment. Our work is open-source and
the code will be at https://maxiuw.github.io/prix.
PRIX, an efficient end-to-end driving model using only camera data, achieves state-of-the-art performance with a Context-aware Recalibration Transformer, outperforming larger multimodal planners in terms of speed and size.
end-to-end autonomous driving models LiDAR sensors BEV feature representations PRIX visual feature extractor generative planning head Context-aware Recalibration Transformer NavSim nuScenes benchmarks multimodal diffusion planners