LLaSO: A Foundational Framework for Reproducible Research in Large
  Language and Speech Model
The development of Large Speech-Language Models (LSLMs) has been slowed by
fragmented architectures and a lack of transparency, hindering the systematic
comparison and reproducibility of research. Unlike in the vision-language
domain, the LSLM field suffers from the common practice of releasing model
weights without their corresponding training data and configurations. To
address these critical gaps, we introduce LLaSO, the first fully open,
end-to-end framework for large-scale speech-language modeling. LLaSO provides
the community with three essential resources: (1) LLaSO-Align, a 12M-instance
speech-text alignment corpus; (2) LLaSO-Instruct, a 13.5M-instance multi-task
instruction-tuning dataset; and (3) LLaSO-Eval, a reproducible benchmark for
standardized evaluation. To validate our framework, we build and release
LLaSO-Base, a 3.8B-parameter reference model trained exclusively on our public
data. It achieves a normalized score of 0.72, establishing a strong,
reproducible baseline that surpasses comparable models. Our analysis reveals
that while broader training coverage enhances performance, significant
generalization gaps persist on unseen tasks, particularly in pure audio
scenarios. By releasing the complete stack of data, benchmarks, and models,
LLaSO establishes a foundational open standard to unify research efforts and
accelerate community-driven progress in LSLMs. We release the code, dataset,
pretrained models, and results in https://github.com/EIT-NLP/LLaSO.
68a862e74fc9195f4f52d1fa
Yirong Sun
claimed_verified
2025-08-22T12:56:40.633Z
65ee8a47ada92a05d3d0339a
/avatars/8c10548192a289001271303bfa32e67d.svg
Yirong Sun
user
YirongSun
68a862e74fc9195f4f52d1fb
Yizhong Geng
68a862e74fc9195f4f52d1fc
Peidong Wei
68a862e74fc9195f4f52d1fd
Yanjun Chen
68a862e74fc9195f4f52d1fe
Jinghan Yang
68a862e74fc9195f4f52d1ff
Rongfei Chen
68a862e74fc9195f4f52d200
Wei Zhang
68a862e74fc9195f4f52d201
Xiaoyu Shen
LLaSO: A Foundational Framework for Reproducible Research in Large
  Language and Speech Model
The development of Large Speech-Language Models (LSLMs) has been slowed by
fragmented architectures and a lack of transparency, hindering the systematic
comparison and reproducibility of research. Unlike in the vision-language
domain, the LSLM field suffers from the common practice of releasing model
weights without their corresponding training data and configurations. To
address these critical gaps, we introduce LLaSO, the first fully open,
end-to-end framework for large-scale speech-language modeling. LLaSO provides
the community with three essential resources: (1) LLaSO-Align, a 12M-instance
speech-text alignment corpus; (2) LLaSO-Instruct, a 13.5M-instance multi-task
instruction-tuning dataset; and (3) LLaSO-Eval, a reproducible benchmark for
standardized evaluation. To validate our framework, we build and release
LLaSO-Base, a 3.8B-parameter reference model trained exclusively on our public
data. It achieves a normalized score of 0.72, establishing a strong,
reproducible baseline that surpasses comparable models. Our analysis reveals
that while broader training coverage enhances performance, significant
generalization gaps persist on unseen tasks, particularly in pure audio
scenarios. By releasing the complete stack of data, benchmarks, and models,
LLaSO establishes a foundational open standard to unify research efforts and
accelerate community-driven progress in LSLMs. We release the code, dataset,
pretrained models, and results in https://github.com/EIT-NLP/LLaSO.
LLaSO is an open framework for large-scale speech-language modeling that provides alignment data, instruction-tuning datasets, and evaluation benchmarks to enhance reproducibility and performance.
Large Speech-Language Models LLaSO LLaSO-Align LLaSO-Instruct LLaSO-Eval speech-text alignment multi-task instruction-tuning reproducible benchmark LLaSO-Base parameter-efficient fine-tuning