YuE: Scaling Open Foundation Models for Long-Form Music Generation
We tackle the task of long-form music generation--particularly the
challenging lyrics-to-song problem--by introducing YuE, a family of
open foundation models based on the LLaMA2 architecture. Specifically, YuE
scales to trillions of tokens and generates up to five minutes of music while
maintaining lyrical alignment, coherent musical structure, and engaging vocal
melodies with appropriate accompaniment. It achieves this through (1)
track-decoupled next-token prediction to overcome dense mixture signals, (2)
structural progressive conditioning for long-context lyrical alignment, and (3)
a multitask, multiphase pre-training recipe to converge and generalize. In
addition, we redesign the in-context learning technique for music generation,
enabling versatile style transfer (e.g., converting Japanese city pop into an
English rap while preserving the original accompaniment) and bidirectional
generation. Through extensive evaluation, we demonstrate that YuE matches or
even surpasses some of the proprietary systems in musicality and vocal agility.
In addition, fine-tuning YuE enables additional controls and enhanced support
for tail languages. Furthermore, beyond generation, we show that YuE's learned
representations can perform well on music understanding tasks, where the
results of YuE match or exceed state-of-the-art methods on the MARBLE
benchmark. Keywords: lyrics2song, song generation, long-form, foundation model,
music generation
67d1027435066eade61549ae
Ruibin Yuan
claimed_verified
2025-03-12T08:36:33.054Z
5fd6f670053c8345eddc1b68
https://cdn-avatars.huggingface.co/v1/production/uploads/5fd6f670053c8345eddc1b68/cuTsu2krRYHC6zYGD2dpQ.jpeg
Ruibin Yuan
user
a43992899
67d1027435066eade61549af
Hanfeng Lin
admin_assigned
2025-03-12T15:28:03.508Z
638df7f3d975533b16a5a739
/avatars/dbc8388e7c32c437b66619ea9b3ad827.svg
Hanfeng Lin
user
agent404
67d1027435066eade61549b0
Shuyue Guo
67d1027435066eade61549b1
Ge Zhang
extracted_confirmed
2025-03-12T06:24:13.961Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
67d1027435066eade61549b2
Jiahao Pan
admin_assigned
2025-03-13T15:41:35.255Z
647f7cf4b0e9676458a014f9
https://cdn-avatars.huggingface.co/v1/production/uploads/647f7cf4b0e9676458a014f9/4L_0MbVErSmPYrk3mIzNP.jpeg
jiahao
user
cherishpjh
67d1027435066eade61549b3
Yongyi Zang
admin_assigned
2025-03-12T15:28:52.965Z
63dda99c0f6d2d6c3efd2362
https://cdn-avatars.huggingface.co/v1/production/uploads/63dda99c0f6d2d6c3efd2362/L5TfrxEWuwWDpR4GeuvI9.png
Yongyi Zang
user
yongyizang
67d1027435066eade61549b4
Haohe Liu
admin_assigned
2025-03-12T15:29:01.345Z
6155245d1c762d4d61b51d5d
https://cdn-avatars.huggingface.co/v1/production/uploads/1677233708065-6155245d1c762d4d61b51d5d.png
haoheliu
user
haoheliu
67d1027435066eade61549b5
Yiming Liang
admin_assigned
2025-03-12T15:29:24.894Z
6555e8d8a0c34cd61a6b9ce3
/avatars/71dc562cef4bd42f6b762f036357c800.svg
yimingliang
user
yimingliang
67d1027435066eade61549b6
Wenye Ma
67d1027435066eade61549b7
Xingjian Du
claimed_verified
2025-03-12T08:36:36.330Z
654907a4a1faff97850c4eff
/avatars/458c90151614bc7f116943b6e67d6b8a.svg
du
user
dododododo
67d1027435066eade61549b8
Xinrun Du
67d1027435066eade61549b9
Zhen Ye
67d1027435066eade61549ba
Tianyu Zheng
67d1027435066eade61549bb
Yinghao Ma
admin_assigned
2025-03-12T15:30:29.451Z
6410665d5364a661bee22524
/avatars/f1cb0e07f36933187ceccbd5dcbeff79.svg
Yinghao Ma
user
nicolaus625
67d1027435066eade61549bc
Minghao Liu
claimed_verified
2025-03-12T08:36:39.193Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
user
Liam-Liu
67d1027435066eade61549bd
Zeyue Tian
claimed_verified
2025-03-16T21:17:50.589Z
6504409b52ca06fef957266d
/avatars/ac96eab7318cd8d472601cf7e7c50c97.svg
Zeyue
user
Zeyue7
67d1027435066eade61549be
Ziya Zhou
67d1027435066eade61549bf
Liumeng Xue
67d1027435066eade61549c0
Xingwei Qu
extracted_pending
2025-03-12T03:41:43.139Z
6628adb14277eae0da5eee28
/avatars/6cb41b80cc5e014e455dfc2a22682e64.svg
HKUST Audio
user
HKUST-Audio
67d1027435066eade61549c1
Yizhi Li
claimed_verified
2025-07-25T08:40:00.600Z
6382252f54421460665ec501
https://cdn-avatars.huggingface.co/v1/production/uploads/6382252f54421460665ec501/gW9fev3T5QPcNq4f9hqB1.jpeg
Yizhi Li
user
yizhilll
67d1027435066eade61549c2
Shangda Wu
67d1027435066eade61549c3
Tianhao Shen
67d1027435066eade61549c4
Ziyang Ma
67d1027435066eade61549c5
Jun Zhan
67d1027435066eade61549c6
Chunhui Wang
admin_assigned
2025-03-12T15:31:34.824Z
66610c2ec629e0e089fc9cf9
/avatars/b3a958de94ba93a8b21ae6d1011f0b45.svg
Chun-Hui Wang
user
ChunHuiWangFN
67d1027435066eade61549c7
Yatian Wang
67d1027435066eade61549c8
Xiaowei Chi
67d1027435066eade61549c9
Xinyue Zhang
67d1027435066eade61549ca
Zhenzhu Yang
67d1027435066eade61549cb
Xiangzhou Wang
67d1027435066eade61549cc
Shansong Liu
claimed_verified
2025-03-28T08:38:25.256Z
64c1f3560a2545084f08350b
https://cdn-avatars.huggingface.co/v1/production/uploads/64c1f3560a2545084f08350b/RbSzN5eFvExrfRLX1Xi_g.jpeg
Shansong Liu
user
dadinghh2
67d1027435066eade61549cd
Lingrui Mei
claimed_verified
2025-06-25T20:59:42.114Z
63120517ae8896941da4c5da
/avatars/10e1be026035f3e24225e6782a710083.svg
Lingrui Mei
user
Chevalier
67d1027435066eade61549ce
Peng Li
67d1027435066eade61549cf
Junjie Wang
claimed_verified
2025-03-12T14:25:45.882Z
62579c55b98dcaa7e0de285d
https://cdn-avatars.huggingface.co/v1/production/uploads/62579c55b98dcaa7e0de285d/0YUd5nloul_bW9yolDGGo.jpeg
wangjunjie
user
wanng
67d1027435066eade61549d0
Jianwei Yu
67d1027435066eade61549d1
Guojian Pang
admin_assigned
2025-03-12T15:34:44.025Z
6454761288c97c8796dcbcee
/avatars/ad39352af72829d687355c92cc8e74a4.svg
Guojian Pang
user
WabbyWabbo
67d1027435066eade61549d2
Xu Li
67d1027435066eade61549d3
Zihao Wang
67d1027435066eade61549d4
Xiaohuan Zhou
admin_assigned
2025-03-12T15:33:44.178Z
668f5875b5b3081d776e4094
/avatars/8c763393f25afbe5fb8b132f775e746a.svg
Xiaohuan Zhou
user
XiaohuanZhou
67d1027435066eade61549d5
Lijun Yu
67d1027435066eade61549d6
Emmanouil Benetos
admin_assigned
2025-03-12T15:34:09.275Z
66b0a0839797b94f001ed874
https://cdn-avatars.huggingface.co/v1/production/uploads/66b0a0839797b94f001ed874/BBtlJdQNyzQX10chnU81s.jpeg
Emmanouil Benetos
user
emmanouilb
67d1027435066eade61549d7
Yong Chen
67d1027435066eade61549d8
Chenghua Lin
admin_assigned
2025-03-12T15:34:33.279Z
6442ba123610a28a4ad5382f
/avatars/97f2d3d2c10058ea41d7bf9087e1f619.svg
Chenghua Lin
user
chenghualin
67d1027435066eade61549d9
Xie Chen
67d1027435066eade61549da
Gus Xia
67d1027435066eade61549db
Zhaoxiang Zhang
admin_assigned
2025-03-12T15:33:32.186Z
6588d5711b44d0e6942e8489
/avatars/f6624b46071e4d1548ea719f26b90b17.svg
zhangzhaoxiang
user
zhangzhaoxiang
67d1027435066eade61549dc
Chao Zhang
67d1027435066eade61549dd
Wenhu Chen
admin_assigned
2025-03-12T15:33:13.185Z
6313a86154e6e5d9f0f94e04
https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
Wenhu Chen
user
wenhu
67d1027435066eade61549de
Xinyu Zhou
67d1027435066eade61549df
Xipeng Qiu
admin_assigned
2025-03-12T15:32:39.488Z
61457b8deff2c9fdb4de4988
https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
Xipeng Qiu
user
xpqiu
67d1027435066eade61549e0
Roger Dannenberg
67d1027435066eade61549e1
Jiaheng Liu
admin_assigned
2025-03-12T15:32:23.034Z
65377c30e48353201e6fdda0
/avatars/a8f803b6f2e598eaee9c52c0d2ddfc16.svg
Jiaheng Liu
user
CheeryLJH
67d1027435066eade61549e2
Jian Yang
67d1027435066eade61549e3
Wenhao Huang
admin_assigned
2025-03-12T15:32:09.795Z
641e5bf65f274a0a92c2f6a2
/avatars/c15a54c51998c0e6367685e8e1737ec9.svg
Wenhao Huang
user
EZ-hwh
67d1027435066eade61549e4
Wei Xue
admin_assigned
2025-03-12T15:31:58.392Z
647403b971f07ae738d4d1dc
/avatars/13a6ff75560c18d857d748cde0feedf3.svg
weixue
user
weixue
67d1027435066eade61549e5
Xu Tan
67d1027435066eade61549e6
Yike Guo
admin_assigned
2025-03-12T15:31:46.078Z
64762b325f70f9b2d0ade28e
/avatars/1f37382724b475ece805f943a8858acd.svg
Yike Guo
user
SuaLily
YuE: Scaling Open Foundation Models for Long-Form Music Generation
We tackle the task of long-form music generation--particularly the
challenging lyrics-to-song problem--by introducing YuE, a family of
open foundation models based on the LLaMA2 architecture. Specifically, YuE
scales to trillions of tokens and generates up to five minutes of music while
maintaining lyrical alignment, coherent musical structure, and engaging vocal
melodies with appropriate accompaniment. It achieves this through (1)
track-decoupled next-token prediction to overcome dense mixture signals, (2)
structural progressive conditioning for long-context lyrical alignment, and (3)
a multitask, multiphase pre-training recipe to converge and generalize. In
addition, we redesign the in-context learning technique for music generation,
enabling versatile style transfer (e.g., converting Japanese city pop into an
English rap while preserving the original accompaniment) and bidirectional
generation. Through extensive evaluation, we demonstrate that YuE matches or
even surpasses some of the proprietary systems in musicality and vocal agility.
In addition, fine-tuning YuE enables additional controls and enhanced support
for tail languages. Furthermore, beyond generation, we show that YuE's learned
representations can perform well on music understanding tasks, where the
results of YuE match or exceed state-of-the-art methods on the MARBLE
benchmark. Keywords: lyrics2song, song generation, long-form, foundation model,
music generation
YuE, a family of open foundation models based on LLaMA2, can generate long-form music with aligned lyrics, coherent structure, and appropriate accompaniment using innovative techniques in next-token prediction, conditioning, and pre-training.
track-decoupled next-token prediction structural progressive conditioning multitask multiphase pre-training in-context learning style transfer bidirectional generation music generation music understanding MARBLE benchmark