Scaling Image Tokenizers with Grouped Spherical Quantization
Vision tokenizers have gained a lot of attraction due to their scalability
and compactness; previous works depend on old-school GAN-based hyperparameters,
biased comparisons, and a lack of comprehensive analysis of the scaling
behaviours. To tackle those issues, we introduce Grouped Spherical Quantization
(GSQ), featuring spherical codebook initialization and lookup regularization to
constrain codebook latent to a spherical surface. Our empirical analysis of
image tokenizer training strategies demonstrates that GSQ-GAN achieves superior
reconstruction quality over state-of-the-art methods with fewer training
iterations, providing a solid foundation for scaling studies. Building on this,
we systematically examine the scaling behaviours of GSQ, specifically in latent
dimensionality, codebook size, and compression ratios, and their impact on
model performance. Our findings reveal distinct behaviours at high and low
spatial compression levels, underscoring challenges in representing
high-dimensional latent spaces. We show that GSQ can restructure
high-dimensional latent into compact, low-dimensional spaces, thus enabling
efficient scaling with improved quality. As a result, GSQ-GAN achieves a 16x
down-sampling with a reconstruction FID (rFID) of 0.50.
674fd182418fad09e5c02283
Jiangtao Wang
674fd182418fad09e5c02284
Zhen Qin
claimed_verified
2024-12-04T09:44:33.644Z
642e63a53c2cf43f6d6dc5ce
/avatars/dfd78c8d55485c22be6e616670a633e5.svg
zhenqin
user
Doreamonzzz
674fd182418fad09e5c02285
Yifan Zhang
claimed_verified
2024-12-04T09:44:35.820Z
647bf082aba7062fe5c51ca9
https://cdn-avatars.huggingface.co/v1/production/uploads/647bf082aba7062fe5c51ca9/p4lY9IjHiWZETKmFq1mtU.jpeg
Yifan Zhang
user
yifAI
674fd182418fad09e5c02286
Vincent Tao Hu
674fd182418fad09e5c02287
Björn Ommer
674fd182418fad09e5c02288
Rania Briq
admin_assigned
2024-12-04T12:01:11.358Z
66f599585754c0c28f211c16
/avatars/ad2e1b76ae203240c99468fbc41e0e16.svg
rania briq
user
briqnn
674fd182418fad09e5c02289
Stefan Kesselheim
admin_assigned
2024-12-04T12:01:05.427Z
63935f43c9d16f6317480184
/avatars/ab3dc8d51f2f9d197a92ae7ea47b9092.svg
Stefan Kesselheim
user
kessel666
Scaling Image Tokenizers with Grouped Spherical Quantization
Vision tokenizers have gained a lot of attraction due to their scalability
and compactness; previous works depend on old-school GAN-based hyperparameters,
biased comparisons, and a lack of comprehensive analysis of the scaling
behaviours. To tackle those issues, we introduce Grouped Spherical Quantization
(GSQ), featuring spherical codebook initialization and lookup regularization to
constrain codebook latent to a spherical surface. Our empirical analysis of
image tokenizer training strategies demonstrates that GSQ-GAN achieves superior
reconstruction quality over state-of-the-art methods with fewer training
iterations, providing a solid foundation for scaling studies. Building on this,
we systematically examine the scaling behaviours of GSQ, specifically in latent
dimensionality, codebook size, and compression ratios, and their impact on
model performance. Our findings reveal distinct behaviours at high and low
spatial compression levels, underscoring challenges in representing
high-dimensional latent spaces. We show that GSQ can restructure
high-dimensional latent into compact, low-dimensional spaces, thus enabling
efficient scaling with improved quality. As a result, GSQ-GAN achieves a 16x
down-sampling with a reconstruction FID (rFID) of 0.50.
Grouped Spherical Quantization (GSQ) improves reconstruction quality and scaling behaviors in vision tokenizers with fewer iterations compared to state-of-the-art methods.
spherical codebook initialization lookup regularization spherical surface GSQ-GAN image tokenizer latent dimensionality codebook size compression ratios high-dimensional latent spaces rFID