GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for
  Autoregressive Image Generation
In autoregressive (AR) image generation, visual tokenizers compress images
into compact discrete latent tokens, enabling efficient training of downstream
autoregressive models for visual generation via next-token prediction. While
scaling visual tokenizers improves image reconstruction quality, it often
degrades downstream generation quality -- a challenge not adequately addressed
in existing literature. To address this, we introduce GigaTok, the first
approach to simultaneously improve image reconstruction, generation, and
representation learning when scaling visual tokenizers. We identify the growing
complexity of latent space as the key factor behind the reconstruction vs.
generation dilemma. To mitigate this, we propose semantic regularization, which
aligns tokenizer features with semantically consistent features from a
pre-trained visual encoder. This constraint prevents excessive latent space
complexity during scaling, yielding consistent improvements in both
reconstruction and downstream autoregressive generation. Building on semantic
regularization, we explore three key practices for scaling tokenizers:(1) using
1D tokenizers for better scalability, (2) prioritizing decoder scaling when
expanding both encoder and decoder, and (3) employing entropy loss to stabilize
training for billion-scale tokenizers. By scaling to 3 space billion
parameters, GigaTok achieves state-of-the-art performance in reconstruction,
downstream AR generation, and downstream AR representation quality.
67fc8e37864dfcbd93d3b802
Tianwei Xiong
claimed_verified
2025-04-14T09:46:29.330Z
668125557b50b433cda2a211
https://cdn-avatars.huggingface.co/v1/production/uploads/668125557b50b433cda2a211/j3z3wT5Rv9IyUKtbzQpnc.png
Tianwei Xiong
user
YuuTennYi
67fc8e37864dfcbd93d3b803
Jun Hao Liew
67fc8e37864dfcbd93d3b804
Zilong Huang
claimed_verified
2025-05-14T07:36:55.605Z
63a430ec84a6a25c65bff20c
/avatars/3ab51214c491f28d8fe6b70c373d72aa.svg
Zilong Huang
user
speedinghzl
67fc8e37864dfcbd93d3b805
Jiashi Feng
67fc8e37864dfcbd93d3b806
Xihui Liu
claimed_verified
2025-04-14T09:46:26.140Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for
  Autoregressive Image Generation
In autoregressive (AR) image generation, visual tokenizers compress images
into compact discrete latent tokens, enabling efficient training of downstream
autoregressive models for visual generation via next-token prediction. While
scaling visual tokenizers improves image reconstruction quality, it often
degrades downstream generation quality -- a challenge not adequately addressed
in existing literature. To address this, we introduce GigaTok, the first
approach to simultaneously improve image reconstruction, generation, and
representation learning when scaling visual tokenizers. We identify the growing
complexity of latent space as the key factor behind the reconstruction vs.
generation dilemma. To mitigate this, we propose semantic regularization, which
aligns tokenizer features with semantically consistent features from a
pre-trained visual encoder. This constraint prevents excessive latent space
complexity during scaling, yielding consistent improvements in both
reconstruction and downstream autoregressive generation. Building on semantic
regularization, we explore three key practices for scaling tokenizers:(1) using
1D tokenizers for better scalability, (2) prioritizing decoder scaling when
expanding both encoder and decoder, and (3) employing entropy loss to stabilize
training for billion-scale tokenizers. By scaling to 3 space billion
parameters, GigaTok achieves state-of-the-art performance in reconstruction,
downstream AR generation, and downstream AR representation quality.
GigaTok improves image reconstruction, generation, and representation quality by scaling visual tokenizers with semantic regularization and specific architectural practices.
autoregressive (AR) image generation visual tokenizers latent tokens image reconstruction downstream generation latent space semantic regularization pre-trained visual encoder 1D tokenizers decoder scaling entropy loss billion-scale tokenizers