MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity
  Reconstruction and Generation
Neural audio codecs have made significant strides in efficiently mapping raw
audio waveforms into discrete token representations, which are foundational for
contemporary audio generative models. However, most existing codecs are
optimized primarily for reconstruction quality, often at the expense of the
downstream modelability of the encoded tokens. Motivated by the need to
overcome this bottleneck, we introduce MagiCodec, a novel
single-layer, streaming Transformer-based audio codec. MagiCodec is designed
with a multistage training pipeline that incorporates Gaussian noise injection
and latent regularization, explicitly targeting the enhancement of semantic
expressiveness in the generated codes while preserving high reconstruction
fidelity. We analytically derive the effect of noise injection in the frequency
domain, demonstrating its efficacy in attenuating high-frequency components and
fostering robust tokenization. Extensive experimental evaluations show that
MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and
downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like
distributions, as observed in natural languages, thereby improving
compatibility with language-model-based generative architectures. The code and
pre-trained models are available at https://github.com/Ereboas/MagiCodec.
683e707763e27c6256f58a51
Yakun Song
683e707763e27c6256f58a52
Jiawei Chen
683e707763e27c6256f58a53
Xiaobin Zhuang
claimed_verified
2025-06-03T08:45:12.616Z
63774ca43a63a2983ffc12f9
https://cdn-avatars.huggingface.co/v1/production/uploads/63774ca43a63a2983ffc12f9/jUJasGf5_2rwiEun_eYl7.png
xiaobin zhuang
user
xiaobinzhuang
683e707763e27c6256f58a54
Chenpeng Du
683e707763e27c6256f58a55
Ziyang Ma
683e707763e27c6256f58a56
Jian Wu
683e707763e27c6256f58a57
Jian Cong
683e707763e27c6256f58a58
Dongya Jia
683e707763e27c6256f58a59
Zhuo Chen
683e707763e27c6256f58a5a
Yuping Wang
683e707763e27c6256f58a5b
Yuxuan Wang
683e707763e27c6256f58a5c
Xie Chen
MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity
  Reconstruction and Generation
Neural audio codecs have made significant strides in efficiently mapping raw
audio waveforms into discrete token representations, which are foundational for
contemporary audio generative models. However, most existing codecs are
optimized primarily for reconstruction quality, often at the expense of the
downstream modelability of the encoded tokens. Motivated by the need to
overcome this bottleneck, we introduce MagiCodec, a novel
single-layer, streaming Transformer-based audio codec. MagiCodec is designed
with a multistage training pipeline that incorporates Gaussian noise injection
and latent regularization, explicitly targeting the enhancement of semantic
expressiveness in the generated codes while preserving high reconstruction
fidelity. We analytically derive the effect of noise injection in the frequency
domain, demonstrating its efficacy in attenuating high-frequency components and
fostering robust tokenization. Extensive experimental evaluations show that
MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and
downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like
distributions, as observed in natural languages, thereby improving
compatibility with language-model-based generative architectures. The code and
pre-trained models are available at https://github.com/Ereboas/MagiCodec.
MagiCodec, a Transformer-based audio codec, enhances semantic tokenization while maintaining high reconstruction quality, improving compatibility with generative models.
Transformer Gaussian noise injection latent regularization frequency domain Zipf-like distributions generative models