StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact
  State Representation
A fundamental challenge in embodied intelligence is developing expressive and
compact state representations for efficient world modeling and decision making.
However, existing methods often fail to achieve this balance, yielding
representations that are either overly redundant or lacking in task-critical
information. We propose an unsupervised approach that learns a highly
compressed two-token state representation using a lightweight encoder and a
pre-trained Diffusion Transformer (DiT) decoder, capitalizing on its strong
generative prior. Our representation is efficient, interpretable, and
integrates seamlessly into existing VLA-based models, improving performance by
14.3% on LIBERO and 30% in real-world task success with minimal inference
overhead. More importantly, we find that the difference between these tokens,
obtained via latent interpolation, naturally serves as a highly effective
latent action, which can be further decoded into executable robot actions. This
emergent capability reveals that our representation captures structured
dynamics without explicit supervision. We name our method StaMo for its ability
to learn generalizable robotic Motion from compact State representation, which
is encoded from static images, challenging the prevalent dependence to learning
latent action on complex architectures and video data. The resulting latent
actions also enhance policy co-training, outperforming prior methods by 10.4%
with improved interpretability. Moreover, our approach scales effectively
across diverse data sources, including real-world robot data, simulation, and
human egocentric video.
68e753a77ae125f9582e6ad4
Mingyu Liu
claimed_verified
2025-10-09T06:49:44.012Z
652e25d2e647b0ee0a024f26
/avatars/b5c65cf6c8d0ddc9b8ef0226e0295d56.svg
Mingyu Liu(SII)
user
MingyuLiu
68e753a77ae125f9582e6ad5
Jiuhe Shu
68e753a77ae125f9582e6ad6
Hui Chen
68e753a77ae125f9582e6ad7
Zeju Li
claimed_verified
2025-10-10T09:30:17.454Z
67d2bf4c94707d574c821076
/avatars/cf143ae7efd3b1b0e10004eb2da70737.svg
Zeju Li
user
dongqianyu99
68e753a77ae125f9582e6ad8
Canyu Zhao
68e753a77ae125f9582e6ad9
Jiange Yang
68e753a77ae125f9582e6ada
Shenyuan Gao
68e753a77ae125f9582e6adb
Hao Chen
68e753a77ae125f9582e6adc
Chunhua Shen
StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact
  State Representation
A fundamental challenge in embodied intelligence is developing expressive and
compact state representations for efficient world modeling and decision making.
However, existing methods often fail to achieve this balance, yielding
representations that are either overly redundant or lacking in task-critical
information. We propose an unsupervised approach that learns a highly
compressed two-token state representation using a lightweight encoder and a
pre-trained Diffusion Transformer (DiT) decoder, capitalizing on its strong
generative prior. Our representation is efficient, interpretable, and
integrates seamlessly into existing VLA-based models, improving performance by
14.3% on LIBERO and 30% in real-world task success with minimal inference
overhead. More importantly, we find that the difference between these tokens,
obtained via latent interpolation, naturally serves as a highly effective
latent action, which can be further decoded into executable robot actions. This
emergent capability reveals that our representation captures structured
dynamics without explicit supervision. We name our method StaMo for its ability
to learn generalizable robotic Motion from compact State representation, which
is encoded from static images, challenging the prevalent dependence to learning
latent action on complex architectures and video data. The resulting latent
actions also enhance policy co-training, outperforming prior methods by 10.4%
with improved interpretability. Moreover, our approach scales effectively
across diverse data sources, including real-world robot data, simulation, and
human egocentric video.
An unsupervised method learns a compact state representation using a lightweight encoder and Diffusion Transformer decoder, improving robotic performance and enabling latent action decoding from static images.
Diffusion Transformer DiT VLA-based models LIBERO latent interpolation latent action StaMo policy co-training human egocentric video