Step-Audio 2 Technical Report
This paper presents Step-Audio~2, an end-to-end multi-modal large language
model designed for industry-strength audio understanding and speech
conversation. By integrating a latent audio encoder and reasoning-centric
reinforcement learning (RL), Step-Audio 2 achieves promising performance in
automatic speech recognition (ASR) and audio understanding. To facilitate
genuine end-to-end speech conversation, Step-Audio 2 incorporates the
generation of discrete audio tokens into language modeling, significantly
enhancing its responsiveness to paralinguistic information such as speaking
styles and emotions. To effectively leverage the rich textual and acoustic
knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented
generation (RAG) and is able to call external tools such as web search to
mitigate hallucination and audio search to switch timbres. Trained on millions
of hours of speech and audio data, Step-Audio 2 delivers intelligence and
expressiveness across diverse conversational scenarios. Evaluation results
demonstrate that Step-Audio 2 achieves state-of-the-art performance on various
audio understanding and conversational benchmarks compared to other open-source
and commercial solutions. Please visit
https://github.com/stepfun-ai/Step-Audio2 for more information.
6880532600f4b5a05f2fbde6
Boyong Wu
6880532600f4b5a05f2fbde7
Chao Yan
claimed_verified
2025-08-29T11:09:44.590Z
67aeb3a4820d941aab225178
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/1BII4nBQiT-rG2kUwXM6V.png
chao yan
user
yanchaomars
6880532600f4b5a05f2fbde8
Chen Hu
claimed_verified
2025-09-01T07:55:22.195Z
648183bfea00b120773198ba
/avatars/b69d04bba11e393d1a59e98653d5cfa6.svg
huchen
user
ccchen1006
6880532600f4b5a05f2fbde9
Cheng Yi
6880532600f4b5a05f2fbdea
Chengli Feng
6880532600f4b5a05f2fbdeb
Fei Tian
6880532600f4b5a05f2fbdec
Feiyu Shen
6880532600f4b5a05f2fbded
Gang Yu
6880532600f4b5a05f2fbdee
Haoyang Zhang
6880532600f4b5a05f2fbdef
Jingbei Li
claimed_verified
2025-07-23T08:36:31.800Z
66b43d78040c500914121ff7
/avatars/585bfc7023c301056027e74e6ff83374.svg
Jingbei Li
user
petronny
6880532600f4b5a05f2fbdf0
Mingrui Chen
6880532600f4b5a05f2fbdf1
Peng Liu
6880532600f4b5a05f2fbdf2
Wang You
6880532600f4b5a05f2fbdf3
Xiangyu Tony Zhang
claimed_verified
2025-08-29T08:46:49.590Z
6524e60c5edccefd13125fd3
/avatars/6daae919255d94f6aa761fb45349650a.svg
Xiangyu
user
Tonyyouyou1
6880532600f4b5a05f2fbdf4
Xingyuan Li
6880532600f4b5a05f2fbdf5
Xuerui Yang
claimed_verified
2025-08-29T09:12:13.270Z
66518fd07d8cb2629a514c18
/avatars/6280b33a6b1532ee938afd4aa303f709.svg
Yang
user
giantPanda0906
6880532600f4b5a05f2fbdf6
Yayue Deng
6880532600f4b5a05f2fbdf7
Yechang Huang
claimed_verified
2025-08-29T08:46:51.232Z
663dabcafc89f85b7aced6b1
/avatars/902511eb952b2bc538bdc8b2b188d06a.svg
Huang Yechang
user
hych2024
6880532600f4b5a05f2fbdf8
Yuxin Li
6880532600f4b5a05f2fbdf9
Yuxin Zhang
claimed_verified
2025-07-23T10:28:22.287Z
680bb50f42bba2f1f7b3e778
/avatars/e91f2773ae67a0cc055ff171e474a5e0.svg
Zhang
user
SmailAA
6880532600f4b5a05f2fbdfa
Zhao You
6880532600f4b5a05f2fbdfb
Brian Li
6880532600f4b5a05f2fbdfc
Changyi Wan
6880532600f4b5a05f2fbdfd
Hanpeng Hu
6880532600f4b5a05f2fbdfe
Jiangjie Zhen
6880532600f4b5a05f2fbdff
Siyu Chen
6880532600f4b5a05f2fbe00
Song Yuan
6880532600f4b5a05f2fbe01
Xuelin Zhang
6880532600f4b5a05f2fbe02
Yimin Jiang
6880532600f4b5a05f2fbe03
Yu Zhou
6880532600f4b5a05f2fbe04
Yuxiang Yang
6880532600f4b5a05f2fbe05
Bingxin Li
6880532600f4b5a05f2fbe06
Buyun Ma
6880532600f4b5a05f2fbe07
Changhe Song
6880532600f4b5a05f2fbe08
Dongqing Pang
6880532600f4b5a05f2fbe09
Guoqiang Hu
claimed_verified
2025-08-29T09:12:14.992Z
65dd5762c33c00ef6169d2e9
https://cdn-avatars.huggingface.co/v1/production/uploads/65dd5762c33c00ef6169d2e9/u_Uh6QT8m1u_j3cD-gAPD.png
Guoqiang Hu
user
airguoqiang
6880532600f4b5a05f2fbe0a
Haiyang Sun
claimed_verified
2025-07-23T08:36:29.790Z
621347acb633fd62c8725494
https://cdn-avatars.huggingface.co/v1/production/uploads/621347acb633fd62c8725494/OEU19lrDklIiMghHODIzv.jpeg
HY
user
Insects
6880532600f4b5a05f2fbe0b
Kang An
6880532600f4b5a05f2fbe0c
Na Wang
6880532600f4b5a05f2fbe0d
Shuli Gao
6880532600f4b5a05f2fbe0e
Wei Ji
6880532600f4b5a05f2fbe0f
Wen Li
6880532600f4b5a05f2fbe10
Wen Sun
6880532600f4b5a05f2fbe11
Xuan Wen
6880532600f4b5a05f2fbe12
Yong Ren
claimed_verified
2025-08-29T08:03:44.999Z
643d27d94816d7cb420a02a1
/avatars/33ed09fae9a6fee9ba8149467a9a04de.svg
Ren Yong
user
y-ren16
6880532600f4b5a05f2fbe13
Yuankai Ma
6880532600f4b5a05f2fbe14
Yufan Lu
6880532600f4b5a05f2fbe15
Bin Wang
6880532600f4b5a05f2fbe16
Bo Li
6880532600f4b5a05f2fbe17
Changxin Miao
6880532600f4b5a05f2fbe18
Che Liu
6880532600f4b5a05f2fbe19
Chen Xu
6880532600f4b5a05f2fbe1a
Dapeng Shi
6880532600f4b5a05f2fbe1b
Dingyuan Hu
6880532600f4b5a05f2fbe1c
Donghang Wu
6880532600f4b5a05f2fbe1d
Enle Liu
6880532600f4b5a05f2fbe1e
Guanzhe Huang
6880532600f4b5a05f2fbe1f
Gulin Yan
6880532600f4b5a05f2fbe20
Han Zhang
6880532600f4b5a05f2fbe21
Hao Nie
6880532600f4b5a05f2fbe22
Haonan Jia
6880532600f4b5a05f2fbe23
Hongyu Zhou
6880532600f4b5a05f2fbe24
Jianjian Sun
6880532600f4b5a05f2fbe25
Jiaoren Wu
6880532600f4b5a05f2fbe26
Jie Wu
6880532600f4b5a05f2fbe27
Jie Yang
6880532600f4b5a05f2fbe28
Jin Yang
6880532600f4b5a05f2fbe29
Junzhe Lin
6880532600f4b5a05f2fbe2a
Kaixiang Li
6880532600f4b5a05f2fbe2b
Lei Yang
6880532600f4b5a05f2fbe2c
Liying Shi
6880532600f4b5a05f2fbe2d
Li Zhou
6880532600f4b5a05f2fbe2e
Longlong Gu
6880532600f4b5a05f2fbe2f
Ming Li
6880532600f4b5a05f2fbe30
Mingliang Li
6880532600f4b5a05f2fbe31
Mingxiao Li
6880532600f4b5a05f2fbe32
Nan Wu
6880532600f4b5a05f2fbe33
Qi Han
6880532600f4b5a05f2fbe34
Qinyuan Tan
6880532600f4b5a05f2fbe35
Shaoliang Pang
6880532600f4b5a05f2fbe36
Shengjie Fan
6880532600f4b5a05f2fbe37
Siqi Liu
6880532600f4b5a05f2fbe38
Tiancheng Cao
6880532600f4b5a05f2fbe39
Wanying Lu
6880532600f4b5a05f2fbe3a
Wenqing He
6880532600f4b5a05f2fbe3b
Wuxun Xie
6880532600f4b5a05f2fbe3c
Xu Zhao
6880532600f4b5a05f2fbe3d
Xueqi Li
6880532600f4b5a05f2fbe3e
Yanbo Yu
6880532600f4b5a05f2fbe3f
Yang Yang
6880532600f4b5a05f2fbe40
Yi Liu
6880532600f4b5a05f2fbe41
Yifan Lu
6880532600f4b5a05f2fbe42
Yilei Wang
6880532600f4b5a05f2fbe43
Yuanhao Ding
6880532600f4b5a05f2fbe44
Yuanwei Liang
6880532600f4b5a05f2fbe45
Yuanwei Lu
6880532600f4b5a05f2fbe46
Yuchu Luo
6880532600f4b5a05f2fbe47
Yuhe Yin
6880532600f4b5a05f2fbe48
Yumeng Zhan
6880532600f4b5a05f2fbe49
Yuxiang Zhang
6880532600f4b5a05f2fbe4a
Zidong Yang
6880532600f4b5a05f2fbe4b
Zixin Zhang
6880532600f4b5a05f2fbe4c
Binxing Jiao
6880532600f4b5a05f2fbe4d
Daxin Jiang
6880532600f4b5a05f2fbe4e
Heung-Yeung Shum
6880532600f4b5a05f2fbe4f
Jiansheng Chen
6880532600f4b5a05f2fbe50
Jing Li
6880532600f4b5a05f2fbe51
Xiangyu Zhang
6880532600f4b5a05f2fbe52
Yibo Zhu
Step-Audio 2 Technical Report
This paper presents Step-Audio~2, an end-to-end multi-modal large language
model designed for industry-strength audio understanding and speech
conversation. By integrating a latent audio encoder and reasoning-centric
reinforcement learning (RL), Step-Audio 2 achieves promising performance in
automatic speech recognition (ASR) and audio understanding. To facilitate
genuine end-to-end speech conversation, Step-Audio 2 incorporates the
generation of discrete audio tokens into language modeling, significantly
enhancing its responsiveness to paralinguistic information such as speaking
styles and emotions. To effectively leverage the rich textual and acoustic
knowledge in real-world data, Step-Audio 2 integrates retrieval-augmented
generation (RAG) and is able to call external tools such as web search to
mitigate hallucination and audio search to switch timbres. Trained on millions
of hours of speech and audio data, Step-Audio 2 delivers intelligence and
expressiveness across diverse conversational scenarios. Evaluation results
demonstrate that Step-Audio 2 achieves state-of-the-art performance on various
audio understanding and conversational benchmarks compared to other open-source
and commercial solutions. Please visit
https://github.com/stepfun-ai/Step-Audio2 for more information.
Step-Audio~2, an end-to-end multi-modal large language model, integrates latent audio encoding and reinforcement learning to achieve state-of-the-art performance in ASR, audio understanding, and speech conversation, incorporating discrete audio token generation and retrieval-augmented generation.
latent audio encoder reasoning-centric reinforcement learning automatic speech recognition discrete audio tokens retrieval-augmented generation audio understanding speech conversation