MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Reliable forecasting of the future behavior of road agents is a critical
component to safe planning in autonomous vehicles. Here, we represent
continuous trajectories as sequences of discrete motion tokens and cast
multi-agent motion prediction as a language modeling task over this domain. Our
model, MotionLM, provides several advantages: First, it does not require
anchors or explicit latent variable optimization to learn multimodal
distributions. Instead, we leverage a single standard language modeling
objective, maximizing the average log probability over sequence tokens. Second,
our approach bypasses post-hoc interaction heuristics where individual agent
trajectory generation is conducted prior to interactive scoring. Instead,
MotionLM produces joint distributions over interactive agent futures in a
single autoregressive decoding process. In addition, the model's sequential
factorization enables temporally causal conditional rollouts. The proposed
approach establishes new state-of-the-art performance for multi-agent motion
prediction on the Waymo Open Motion Dataset, ranking 1st on the interactive
challenge leaderboard.
65162e608fe117b68e33934a
Ari Seff
65162e608fe117b68e33934b
Brian Cera
65162e608fe117b68e33934c
Dian Chen
65162e608fe117b68e33934d
Mason Ng
65162e608fe117b68e33934e
Aurick Zhou
65162e608fe117b68e33934f
Nigamaa Nayakanti
65162e608fe117b68e339350
Khaled S. Refaat
65162e608fe117b68e339351
Rami Al-Rfou
claimed_verified
2023-10-02T06:59:48.552Z
6449a9cfeb7db8f70fb7f798
/avatars/a4a5a96e9c857f44d43456579cddc51a.svg
Rami Al-Rfou
user
rmyeid
65162e608fe117b68e339352
Benjamin Sapp
MotionLM: Multi-Agent Motion Forecasting as Language Modeling
Reliable forecasting of the future behavior of road agents is a critical
component to safe planning in autonomous vehicles. Here, we represent
continuous trajectories as sequences of discrete motion tokens and cast
multi-agent motion prediction as a language modeling task over this domain. Our
model, MotionLM, provides several advantages: First, it does not require
anchors or explicit latent variable optimization to learn multimodal
distributions. Instead, we leverage a single standard language modeling
objective, maximizing the average log probability over sequence tokens. Second,
our approach bypasses post-hoc interaction heuristics where individual agent
trajectory generation is conducted prior to interactive scoring. Instead,
MotionLM produces joint distributions over interactive agent futures in a
single autoregressive decoding process. In addition, the model's sequential
factorization enables temporally causal conditional rollouts. The proposed
approach establishes new state-of-the-art performance for multi-agent motion
prediction on the Waymo Open Motion Dataset, ranking 1st on the interactive
challenge leaderboard.
MotionLM, a language modeling approach, forecasts multi-agent motion by generating joint distributions over interactive agent futures, achieving state-of-the-art performance on the Waymo Open Motion Dataset.
MotionLM language modeling multimodal distributions autoregressive decoding temporally causal conditional rollouts multi-agent motion prediction Waymo Open Motion Dataset