CODA: Repurposing Continuous VAEs for Discrete Tokenization
Discrete visual tokenizers transform images into a sequence of tokens,
enabling token-based visual generation akin to language models. However, this
process is inherently challenging, as it requires both compressing visual
signals into a compact representation and discretizing them into a fixed set of
codes. Traditional discrete tokenizers typically learn the two tasks jointly,
often leading to unstable training, low codebook utilization, and limited
reconstruction quality. In this paper, we introduce
CODA(COntinuous-to-Discrete Adaptation), a
framework that decouples compression and discretization. Instead of training
discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs --
already optimized for perceptual compression -- into discrete tokenizers via a
carefully designed discretization process. By primarily focusing on
discretization, CODA ensures stable and efficient training while retaining the
strong visual fidelity of continuous VAEs. Empirically, with 6
times less training budget than standard VQGAN, our approach achieves a
remarkable codebook utilization of 100% and notable reconstruction FID (rFID)
of 0.43 and 1.34 for 8 times and 16 times
compression on ImageNet 256times 256 benchmark.
67e2b2c1a46a0939c33119b5
Zeyu Liu
claimed_verified
2025-03-25T14:35:58.054Z
64f2a14396a5ffb820ab39a8
/avatars/3d7c3be17c53eb52bd8a75d3c51e6891.svg
Zeyu Liu
user
shuidi0020
67e2b2c1a46a0939c33119b6
Zanlin Ni
67e2b2c1a46a0939c33119b7
Yeguo Hua
67e2b2c1a46a0939c33119b8
Xin Deng
67e2b2c1a46a0939c33119b9
Xiao Ma
67e2b2c1a46a0939c33119ba
Cheng Zhong
67e2b2c1a46a0939c33119bb
Gao Huang
CODA: Repurposing Continuous VAEs for Discrete Tokenization
Discrete visual tokenizers transform images into a sequence of tokens,
enabling token-based visual generation akin to language models. However, this
process is inherently challenging, as it requires both compressing visual
signals into a compact representation and discretizing them into a fixed set of
codes. Traditional discrete tokenizers typically learn the two tasks jointly,
often leading to unstable training, low codebook utilization, and limited
reconstruction quality. In this paper, we introduce
CODA(COntinuous-to-Discrete Adaptation), a
framework that decouples compression and discretization. Instead of training
discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs --
already optimized for perceptual compression -- into discrete tokenizers via a
carefully designed discretization process. By primarily focusing on
discretization, CODA ensures stable and efficient training while retaining the
strong visual fidelity of continuous VAEs. Empirically, with 6
times less training budget than standard VQGAN, our approach achieves a
remarkable codebook utilization of 100% and notable reconstruction FID (rFID)
of 0.43 and 1.34 for 8 times and 16 times
compression on ImageNet 256times 256 benchmark.
CODA adaptively transforms continuous VAEs into discrete tokenizers, achieving high codebook utilization and quality with significantly reduced training time.
discrete visual tokenizers token-based visual generation language models compression discretization continuous VAEs perceptual compression codebook utilization reconstruction FID ImageNet benchmark