FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocoder
The goal of this paper is to generate realistic audio with a lightweight and
fast diffusion-based vocoder, named FreGrad. Our framework consists of the
following three key components: (1) We employ discrete wavelet transform that
decomposes a complicated waveform into sub-band wavelets, which helps FreGrad
to operate on a simple and concise feature space, (2) We design a
frequency-aware dilated convolution that elevates frequency awareness,
resulting in generating speech with accurate frequency information, and (3) We
introduce a bag of tricks that boosts the generation quality of the proposed
model. In our experiments, FreGrad achieves 3.7 times faster training time and
2.2 times faster inference speed compared to our baseline while reducing the
model size by 0.6 times (only 1.78M parameters) without sacrificing the output
quality. Audio samples are available at:
https://mm.kaist.ac.kr/projects/FreGrad.
65a9e92668139e3c423d0fc2
Tan Dat Nguyen
admin_assigned
2024-01-19T13:35:56.495Z
64b0edb86c5beff5c5a9ed31
/avatars/9e96c56be223e42cc30a6103661fdfe9.svg
Tan Dat Nguyen
user
ntdatvn
65a9e92668139e3c423d0fc3
Ji-Hoon Kim
65a9e92668139e3c423d0fc4
Youngjoon Jang
admin_assigned
2024-01-19T13:36:20.927Z
65a4c4ed2548c41ad9b1421c
https://cdn-avatars.huggingface.co/v1/production/uploads/65a4c4ed2548c41ad9b1421c/bMQbowjHKvq-bKpzalvWm.jpeg
Youngjoon Jang
user
yjoonjang
65a9e92668139e3c423d0fc5
Jaehun Kim
admin_assigned
2024-01-19T13:36:27.948Z
6213510ea82164d119812be3
/avatars/e4802350e3bcc3688e7b2ecb667bf430.svg
Jaehun Kim
user
smoked22
65a9e92668139e3c423d0fc6
Joon Son Chung
admin_assigned
2024-01-19T13:36:35.286Z
647dcc87e4d52fe0e01c8ffc
/avatars/56fbf5604dc4082b0c494bd49f345c71.svg
Joon Son Chung
user
joonson
FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocoder
The goal of this paper is to generate realistic audio with a lightweight and
fast diffusion-based vocoder, named FreGrad. Our framework consists of the
following three key components: (1) We employ discrete wavelet transform that
decomposes a complicated waveform into sub-band wavelets, which helps FreGrad
to operate on a simple and concise feature space, (2) We design a
frequency-aware dilated convolution that elevates frequency awareness,
resulting in generating speech with accurate frequency information, and (3) We
introduce a bag of tricks that boosts the generation quality of the proposed
model. In our experiments, FreGrad achieves 3.7 times faster training time and
2.2 times faster inference speed compared to our baseline while reducing the
model size by 0.6 times (only 1.78M parameters) without sacrificing the output
quality. Audio samples are available at:
https://mm.kaist.ac.kr/projects/FreGrad.
FreGrad, a lightweight and fast diffusion-based vocoder, uses discrete wavelet transform, frequency-aware dilated convolution, and a bag of tricks to generate high-quality audio with improved training and inference speeds.
discrete wavelet transform sub-band wavelets frequency-aware dilated convolution diffusion-based vocoder