MuCodec: Ultra Low-Bitrate Music Codec
Music codecs are a vital aspect of audio codec research, and ultra
low-bitrate compression holds significant importance for music transmission and
generation. Due to the complexity of music backgrounds and the richness of
vocals, solely relying on modeling semantic or acoustic information cannot
effectively reconstruct music with both vocals and backgrounds. To address this
issue, we propose MuCodec, specifically targeting music compression and
reconstruction tasks at ultra low bitrates. MuCodec employs MuEncoder to
extract both acoustic and semantic features, discretizes them with RVQ, and
obtains Mel-VAE features via flow-matching. The music is then reconstructed
using a pre-trained MEL-VAE decoder and HiFi-GAN. MuCodec can reconstruct
high-fidelity music at ultra low (0.35kbps) or high bitrates (1.35kbps),
achieving the best results to date in both subjective and objective metrics.
Code and Demo: https://xuyaoxun.github.io/MuCodec_demo/.
66f0c6b4831eacfcad5a970b
Yaoxun Xu
admin_assigned
2024-09-23T07:32:36.922Z
642fd482450c0de9a1d1c8b5
/avatars/3765b121feb979e410f665bf20191161.svg
yaoxunxu
user
yaoxunxu
66f0c6b4831eacfcad5a970c
Hangting Chen
admin_assigned
2024-09-23T07:32:42.807Z
631b2dbd0f22997ce6728e27
/avatars/84dc35edbda6c071fcecec01918c5c56.svg
Hangting Chen
user
hangtingchen
66f0c6b4831eacfcad5a970d
Jianwei Yu
66f0c6b4831eacfcad5a970e
Wei Tan
claimed_verified
2025-06-18T12:20:22.882Z
68380d27f1a2d15e71df5ea8
/avatars/181f721bdb0c0b84f61ce4f851953fa6.svg
way tan
user
waytan22
66f0c6b4831eacfcad5a970f
Rongzhi Gu
66f0c6b4831eacfcad5a9710
Shun Lei
admin_assigned
2024-09-23T07:33:26.089Z
6531768050f0ed02addafa95
/avatars/b6689d42ab2f994ba0f149fba7732261.svg
Shun Lei
user
lglg666
66f0c6b4831eacfcad5a9711
Zhiwei Lin
66f0c6b4831eacfcad5a9712
Zhiyong Wu
admin_assigned
2024-09-23T07:34:09.118Z
6280e830e99dccaac4bbfde5
/avatars/9242b8d2826ce2f79af9bb794bba2b61.svg
Zhiyong Wu
user
zy001
MuCodec: Ultra Low-Bitrate Music Codec
Music codecs are a vital aspect of audio codec research, and ultra
low-bitrate compression holds significant importance for music transmission and
generation. Due to the complexity of music backgrounds and the richness of
vocals, solely relying on modeling semantic or acoustic information cannot
effectively reconstruct music with both vocals and backgrounds. To address this
issue, we propose MuCodec, specifically targeting music compression and
reconstruction tasks at ultra low bitrates. MuCodec employs MuEncoder to
extract both acoustic and semantic features, discretizes them with RVQ, and
obtains Mel-VAE features via flow-matching. The music is then reconstructed
using a pre-trained MEL-VAE decoder and HiFi-GAN. MuCodec can reconstruct
high-fidelity music at ultra low (0.35kbps) or high bitrates (1.35kbps),
achieving the best results to date in both subjective and objective metrics.
Code and Demo: https://xuyaoxun.github.io/MuCodec_demo/.
MuCodec, using MuEncoder, RVQ, flow-matching, MEL-VAE, and HiFi-GAN, achieves high-fidelity music reconstruction at ultra-low bitrates.
MuEncoder RVQ flow-matching MEL-VAE HiFi-GAN