ε-VAE: Denoising as Visual Decoding
In generative modeling, tokenization simplifies complex data into compact,
structured representations, creating a more efficient, learnable space. For
high-dimensional visual data, it reduces redundancy and emphasizes key features
for high-quality generation. Current visual tokenization methods rely on a
traditional autoencoder framework, where the encoder compresses data into
latent representations, and the decoder reconstructs the original input. In
this work, we offer a new perspective by proposing denoising as decoding,
shifting from single-step reconstruction to iterative refinement. Specifically,
we replace the decoder with a diffusion process that iteratively refines noise
to recover the original image, guided by the latents provided by the encoder.
We evaluate our approach by assessing both reconstruction (rFID) and generation
quality (FID), comparing it to state-of-the-art autoencoding approach. We hope
this work offers new insights into integrating iterative generation and
autoencoding for improved compression and generation.
6706ec61981508d3eb268d4e
Long Zhao
claimed_verified
2024-10-10T08:01:55.932Z
650c249887dcda6616baa040
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tIVfxoAHAJ0sWyDMkaarA.jpeg
Long Zhao
user
garyzhao9012
6706ec61981508d3eb268d4f
Sanghyun Woo
6706ec61981508d3eb268d50
Ziyu Wan
6706ec61981508d3eb268d51
Yandong Li
6706ec61981508d3eb268d52
Han Zhang
6706ec61981508d3eb268d53
Boqing Gong
6706ec61981508d3eb268d54
Hartwig Adam
6706ec61981508d3eb268d55
Xuhui Jia
6706ec61981508d3eb268d56
Ting Liu
ε-VAE: Denoising as Visual Decoding
In generative modeling, tokenization simplifies complex data into compact,
structured representations, creating a more efficient, learnable space. For
high-dimensional visual data, it reduces redundancy and emphasizes key features
for high-quality generation. Current visual tokenization methods rely on a
traditional autoencoder framework, where the encoder compresses data into
latent representations, and the decoder reconstructs the original input. In
this work, we offer a new perspective by proposing denoising as decoding,
shifting from single-step reconstruction to iterative refinement. Specifically,
we replace the decoder with a diffusion process that iteratively refines noise
to recover the original image, guided by the latents provided by the encoder.
We evaluate our approach by assessing both reconstruction (rFID) and generation
quality (FID), comparing it to state-of-the-art autoencoding approach. We hope
this work offers new insights into integrating iterative generation and
autoencoding for improved compression and generation.
Proposes a novel visual tokenization method that uses a diffusion process for iterative refinement in the decoding stage, improving image generation and reconstruction quality.
tokenization generative modeling high-dimensional visual data autoencoder framework denoising diffusion process iterative refinement rFID FID