Bridging Continuous and Discrete Tokens for Autoregressive Visual
  Generation
Autoregressive visual generation models typically rely on tokenizers to
compress images into tokens that can be predicted sequentially. A fundamental
dilemma exists in token representation: discrete tokens enable straightforward
modeling with standard cross-entropy loss, but suffer from information loss and
tokenizer training instability; continuous tokens better preserve visual
details, but require complex distribution modeling, complicating the generation
pipeline. In this paper, we propose TokenBridge, which bridges this gap by
maintaining the strong representation capacity of continuous tokens while
preserving the modeling simplicity of discrete tokens. To achieve this, we
decouple discretization from the tokenizer training process through
post-training quantization that directly obtains discrete tokens from
continuous representations. Specifically, we introduce a dimension-wise
quantization strategy that independently discretizes each feature dimension,
paired with a lightweight autoregressive prediction mechanism that efficiently
model the resulting large token space. Extensive experiments show that our
approach achieves reconstruction and generation quality on par with continuous
methods while using standard categorical prediction. This work demonstrates
that bridging discrete and continuous paradigms can effectively harness the
strengths of both approaches, providing a promising direction for high-quality
visual generation with simple autoregressive modeling. Project page:
https://yuqingwang1029.github.io/TokenBridge.
67e0bd81b04d9e836829c468
Yuqing Wang
claimed_verified
2025-03-24T08:10:42.267Z
63ea23b9dedfeebe54d02bdf
/avatars/4d9f9a546aa8c63e277161ea700075c4.svg
Yuqing Wang
user
Epiphqny
67e0bd81b04d9e836829c469
Zhijie Lin
admin_assigned
2025-03-24T12:40:54.208Z
64415957bd0c9726529802f6
/avatars/1132d1ee68fb58ec635d57c8175caacd.svg
Zhijie Lin
user
Ikuinen
67e0bd81b04d9e836829c46a
Yao Teng
admin_assigned
2025-03-24T12:41:01.459Z
6427e08288215cee63b1c44d
https://cdn-avatars.huggingface.co/v1/production/uploads/6427e08288215cee63b1c44d/rzaG978FF-ywzicWNl_xl.jpeg
yao teng
user
tytyt
67e0bd81b04d9e836829c46b
Yuanzhi Zhu
admin_assigned
2025-03-24T12:41:07.856Z
627d2723401f42c57b6b7c0c
/avatars/6ff754e56aaee63d8572881a6a966171.svg
Yuanzhi Zhu
user
Yuanzhi
67e0bd81b04d9e836829c46c
Shuhuai Ren
claimed_verified
2025-03-24T08:10:44.045Z
60d2e681b8448e1785bbda06
https://cdn-avatars.huggingface.co/v1/production/uploads/1624434302056-noauth.jpeg
Shuhuai Ren
user
ShuhuaiRen
67e0bd81b04d9e836829c46d
Jiashi Feng
admin_assigned
2025-03-24T12:41:16.160Z
67298e44017b96a1d0101dc4
/avatars/1f8ed1a3e911e6a3021087b9371d284c.svg
Jiashi Feng
user
jshfeng
67e0bd81b04d9e836829c46e
Xihui Liu
admin_assigned
2025-03-24T12:41:22.229Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
Bridging Continuous and Discrete Tokens for Autoregressive Visual
  Generation
Autoregressive visual generation models typically rely on tokenizers to
compress images into tokens that can be predicted sequentially. A fundamental
dilemma exists in token representation: discrete tokens enable straightforward
modeling with standard cross-entropy loss, but suffer from information loss and
tokenizer training instability; continuous tokens better preserve visual
details, but require complex distribution modeling, complicating the generation
pipeline. In this paper, we propose TokenBridge, which bridges this gap by
maintaining the strong representation capacity of continuous tokens while
preserving the modeling simplicity of discrete tokens. To achieve this, we
decouple discretization from the tokenizer training process through
post-training quantization that directly obtains discrete tokens from
continuous representations. Specifically, we introduce a dimension-wise
quantization strategy that independently discretizes each feature dimension,
paired with a lightweight autoregressive prediction mechanism that efficiently
model the resulting large token space. Extensive experiments show that our
approach achieves reconstruction and generation quality on par with continuous
methods while using standard categorical prediction. This work demonstrates
that bridging discrete and continuous paradigms can effectively harness the
strengths of both approaches, providing a promising direction for high-quality
visual generation with simple autoregressive modeling. Project page:
https://yuqingwang1029.github.io/TokenBridge.
TokenBridge addresses the dilemma between discrete and continuous token representations in autoregressive visual generation by using post-training quantization to preserve visual details while maintaining modeling simplicity.
autoregressive visual generation models tokenizers discrete tokens continuous tokens cross-entropy loss distribution modeling TokenBridge dimension-wise quantization lightweight autoregressive prediction mechanism