AToken: A Unified Tokenizer for Vision
We present AToken, the first unified visual tokenizer that achieves both
high-fidelity reconstruction and semantic understanding across images, videos,
and 3D assets. Unlike existing tokenizers that specialize in either
reconstruction or understanding for single modalities, AToken encodes these
diverse visual inputs into a shared 4D latent space, unifying both tasks and
modalities in a single framework. Specifically, we introduce a pure transformer
architecture with 4D rotary position embeddings to process visual inputs of
arbitrary resolutions and temporal durations. To ensure stable training, we
introduce an adversarial-free training objective that combines perceptual and
Gram matrix losses, achieving state-of-the-art reconstruction quality. By
employing a progressive training curriculum, AToken gradually expands from
single images, videos, and 3D, and supports both continuous and discrete latent
tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01
rFVD with 32.6% MSRVTT retrieval for videos, and 28.19 PSNR with 90.9%
classification accuracy for 3D. In downstream applications, AToken enables both
visual generation tasks (e.g., image generation with continuous and discrete
tokens, text-to-video generation, image-to-3D synthesis) and understanding
tasks (e.g., multimodal LLMs), achieving competitive performance across all
benchmarks. These results shed light on the next-generation multimodal AI
systems built upon unified visual tokenization.
68ccb6a63df9ac65e93dc603
Jiasen Lu
admin_assigned
2025-09-19T10:41:09.966Z
62b6b0397523238923221df9
/avatars/77068771dd51df7519516cd502a88789.svg
Jiasenlu
user
Jiasenlu
68ccb6a63df9ac65e93dc604
Liangchen Song
claimed_verified
2025-09-21T13:13:04.072Z
63d9ea2aaefaa7352587151c
/avatars/2f02ee1b2a0018c74ef1d47883782bd5.svg
Liangchen Song
user
lsongx
68ccb6a63df9ac65e93dc605
Mingze Xu
claimed_verified
2025-09-23T15:15:39.815Z
65fc655930aee594d74dd7b8
/avatars/e8df7170fa1cf99bb3a2b9a083ee99a1.svg
Mingze Xu
user
xumingze0308
68ccb6a63df9ac65e93dc606
Byeongjoo Ahn
admin_assigned
2025-09-19T10:41:32.933Z
673c383c0a4d127a7f6482d5
/avatars/5f875477a721204c5e3d67c391dfcea4.svg
Byeongjoo Ahn
user
byeongjooahn
68ccb6a63df9ac65e93dc607
Yanjun Wang
68ccb6a63df9ac65e93dc608
Chen Chen
68ccb6a63df9ac65e93dc609
Afshin Dehghan
admin_assigned
2025-09-19T10:42:19.278Z
684573fcf072c657cac88078
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OFfIBultMLWfVY9UPlBVK.png
afshin dehghan
user
afshin525
68ccb6a63df9ac65e93dc60a
Yinfei Yang
AToken: A Unified Tokenizer for Vision
We present AToken, the first unified visual tokenizer that achieves both
high-fidelity reconstruction and semantic understanding across images, videos,
and 3D assets. Unlike existing tokenizers that specialize in either
reconstruction or understanding for single modalities, AToken encodes these
diverse visual inputs into a shared 4D latent space, unifying both tasks and
modalities in a single framework. Specifically, we introduce a pure transformer
architecture with 4D rotary position embeddings to process visual inputs of
arbitrary resolutions and temporal durations. To ensure stable training, we
introduce an adversarial-free training objective that combines perceptual and
Gram matrix losses, achieving state-of-the-art reconstruction quality. By
employing a progressive training curriculum, AToken gradually expands from
single images, videos, and 3D, and supports both continuous and discrete latent
tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01
rFVD with 32.6% MSRVTT retrieval for videos, and 28.19 PSNR with 90.9%
classification accuracy for 3D. In downstream applications, AToken enables both
visual generation tasks (e.g., image generation with continuous and discrete
tokens, text-to-video generation, image-to-3D synthesis) and understanding
tasks (e.g., multimodal LLMs), achieving competitive performance across all
benchmarks. These results shed light on the next-generation multimodal AI
systems built upon unified visual tokenization.
AToken, a unified visual tokenizer, achieves high-fidelity reconstruction and semantic understanding across images, videos, and 3D assets using a 4D transformer architecture with adversarial-free training.
unified visual tokenizer 4D latent space pure transformer architecture 4D rotary position embeddings adversarial-free training objective perceptual loss Gram matrix loss progressive training curriculum continuous latent tokens discrete latent tokens rFID ImageNet accuracy rFVD MSRVTT retrieval PSNR classification accuracy visual generation tasks text-to-video generation image-to-3D synthesis multimodal LLMs