Kimi-Audio Technical Report
We present Kimi-Audio, an open-source audio foundation model that excels in
audio understanding, generation, and conversation. We detail the practices in
building Kimi-Audio, including model architecture, data curation, training
recipe, inference deployment, and evaluation. Specifically, we leverage a
12.5Hz audio tokenizer, design a novel LLM-based architecture with continuous
features as input and discrete tokens as output, and develop a chunk-wise
streaming detokenizer based on flow matching. We curate a pre-training dataset
that consists of more than 13 million hours of audio data covering a wide range
of modalities including speech, sound, and music, and build a pipeline to
construct high-quality and diverse post-training data. Initialized from a
pre-trained LLM, Kimi-Audio is continual pre-trained on both audio and text
data with several carefully designed tasks, and then fine-tuned to support a
diverse of audio-related tasks. Extensive evaluation shows that Kimi-Audio
achieves state-of-the-art performance on a range of audio benchmarks including
speech recognition, audio understanding, audio question answering, and speech
conversation. We release the codes, model checkpoints, as well as the
evaluation toolkits in https://github.com/MoonshotAI/Kimi-Audio.
680f790454e7aa920953b2c2
KimiTeam
680f790454e7aa920953b2c3
Ding Ding
680f790454e7aa920953b2c4
Zeqian Ju
admin_assigned
2025-04-29T12:38:29.005Z
668fb0cda4d0df7e51653d96
/avatars/282fc2f3113f71118c51b5f4c2ec0693.svg
zeqian ju
user
jzq11111
680f790454e7aa920953b2c5
Yichong Leng
admin_assigned
2025-04-29T12:38:43.918Z
64a0347b528a9bbe59d6e08c
/avatars/6dd0bad84d711d1048a0a4169e621773.svg
Yichong Leng
user
ustcscallion
680f790454e7aa920953b2c6
Songxiang Liu
admin_assigned
2025-04-29T12:38:58.251Z
626b4350822e3d85324c6054
/avatars/3e27115ea73f33ba40e8599cdf2725af.svg
liusongxiang
user
liusongxiang
680f790454e7aa920953b2c7
Tong Liu
680f790454e7aa920953b2c8
Zeyu Shang
admin_assigned
2025-04-29T12:39:22.417Z
643cf7aacc862a73634ce4fb
/avatars/084ae611d4be29a2dad5b806c221448b.svg
shangzeyu
user
shangzy
680f790454e7aa920953b2c9
Kai Shen
680f790454e7aa920953b2ca
Wei Song
680f790454e7aa920953b2cb
Xu Tan
admin_assigned
2025-04-29T12:39:35.289Z
5f1040b6e9d71719e3be71d2
/avatars/a2f28940236ae625ed3810ad62e343ff.svg
Xu Tan
user
xutan
680f790454e7aa920953b2cc
Heyi Tang
680f790454e7aa920953b2cd
Zhengtao Wang
admin_assigned
2025-04-29T12:39:54.936Z
65a3497bcb2c2799acb73d2a
/avatars/bb6aaae76b145a2cdae802debe1aba00.svg
zhengtao
user
wangzhengtao
680f790454e7aa920953b2ce
Chu Wei
680f790454e7aa920953b2cf
Yifei Xin
admin_assigned
2025-04-29T12:40:02.675Z
62d34b1d5d683b8756d6f846
/avatars/c6cdb239f9f9dbd698423066d25af1b3.svg
Yifei Xin
user
zolars
680f790454e7aa920953b2d0
Xinran Xu
admin_assigned
2025-04-29T12:40:13.967Z
6806011c332f406ba175eb27
/avatars/d09990175f05a8c51b0bfa56ad5b8d75.svg
Xu Xinran
user
Xinnnnnar
680f790454e7aa920953b2d1
Jianwei Yu
680f790454e7aa920953b2d2
Yutao Zhang
680f790454e7aa920953b2d3
Xinyu Zhou
680f790454e7aa920953b2d4
Y. Charles
680f790454e7aa920953b2d5
Jun Chen
680f790454e7aa920953b2d6
Yanru Chen
680f790454e7aa920953b2d7
Yulun Du
admin_assigned
2025-04-29T12:35:52.443Z
6340f31fb78ed99eab04ce33
/avatars/2e7fcbf0233bdc0bc9a3f4603fd8bf90.svg
Du
user
Yulun
680f790454e7aa920953b2d8
Weiran He
680f790454e7aa920953b2d9
Zhenxing Hu
680f790454e7aa920953b2da
Guokun Lai
680f790454e7aa920953b2db
Qingcheng Li
680f790454e7aa920953b2dc
Yangyang Liu
680f790454e7aa920953b2dd
Weidong Sun
680f790454e7aa920953b2de
Jianzhou Wang
680f790454e7aa920953b2df
Yuzhi Wang
admin_assigned
2025-04-29T12:41:00.101Z
67127a470a82509269d738ae
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/M9qLmI3P6dT2FIwEPFJq0.png
yuzhi wang
user
vin-tage
680f790454e7aa920953b2e0
Yuefeng Wu
680f790454e7aa920953b2e1
Yuxin Wu
680f790454e7aa920953b2e2
Dongchao Yang
admin_assigned
2025-04-29T12:37:05.161Z
63c7636b656e7822e23e6f6b
/avatars/41bfd5e1ce6daab6058eacfd33c7a268.svg
Dongchao Yang
user
Dongchao
680f790454e7aa920953b2e3
Hao Yang
680f790454e7aa920953b2e4
Ying Yang
680f790454e7aa920953b2e5
Zhilin Yang
admin_assigned
2025-04-29T12:36:54.887Z
64bf74154d2052b1aa5ca6d9
/avatars/7aa6f2952cdbc20cfa758fdd905f06a6.svg
ZHILIN YANG
user
bruceyannnn
680f790454e7aa920953b2e6
Aoxiong Yin
admin_assigned
2025-04-29T12:36:35.158Z
66ed29833f928a11d8773df8
/avatars/ab2f94eefa54f62f486efc7454e508a6.svg
aoxiong
user
yinaoxiong
680f790454e7aa920953b2e7
Ruibin Yuan
admin_assigned
2025-04-29T12:36:22.098Z
5fd6f670053c8345eddc1b68
https://cdn-avatars.huggingface.co/v1/production/uploads/5fd6f670053c8345eddc1b68/cuTsu2krRYHC6zYGD2dpQ.jpeg
Ruibin Yuan
user
a43992899
680f790454e7aa920953b2e8
Yutong Zhang
680f790454e7aa920953b2e9
Zaida Zhou
admin_assigned
2025-04-29T12:36:03.786Z
64409d69518271b0d1c033a6
/avatars/c79eb36c4ad96286afda834e260a1c09.svg
zhouzaida
user
zhouzaida
Kimi-Audio Technical Report
We present Kimi-Audio, an open-source audio foundation model that excels in
audio understanding, generation, and conversation. We detail the practices in
building Kimi-Audio, including model architecture, data curation, training
recipe, inference deployment, and evaluation. Specifically, we leverage a
12.5Hz audio tokenizer, design a novel LLM-based architecture with continuous
features as input and discrete tokens as output, and develop a chunk-wise
streaming detokenizer based on flow matching. We curate a pre-training dataset
that consists of more than 13 million hours of audio data covering a wide range
of modalities including speech, sound, and music, and build a pipeline to
construct high-quality and diverse post-training data. Initialized from a
pre-trained LLM, Kimi-Audio is continual pre-trained on both audio and text
data with several carefully designed tasks, and then fine-tuned to support a
diverse of audio-related tasks. Extensive evaluation shows that Kimi-Audio
achieves state-of-the-art performance on a range of audio benchmarks including
speech recognition, audio understanding, audio question answering, and speech
conversation. We release the codes, model checkpoints, as well as the
evaluation toolkits in https://github.com/MoonshotAI/Kimi-Audio.
Kimi-Audio, an open-source audio foundation model, achieves state-of-the-art performance across audio-related tasks through a novel LLM-based architecture and comprehensive training and evaluation processes.
LLM-based architecture continuous features discrete tokens flow matching chunk-wise streaming detokenizer pre-training dataset post-training data continual pre-training fine-tuning speech recognition audio understanding audio question answering speech conversation