Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully
  Open MLLMs
Fully open multimodal large language models (MLLMs) currently lag behind
proprietary counterparts, primarily due to a significant gap in data quality
for supervised fine-tuning (SFT). Existing open-source datasets are often
plagued by widespread noise and a critical deficit in complex reasoning data,
such as Chain-of-Thought (CoT), which hinders the development of advanced model
capabilities. Addressing these challenges, our work makes three primary
contributions. First, we introduce Honey-Data-15M, a new SFT dataset comprising
approximately 15 million QA pairs, processed through multiple cleaning
techniques and enhanced with a novel dual-level (short and long) CoT enrichment
strategy. Second, we introduce HoneyPipe, the data curation pipeline, and its
underlying framework DataStudio, providing the community with a transparent and
adaptable methodology for data curation that moves beyond static dataset
releases. Finally, to validate our dataset and pipeline, we train Bee-8B, an 8B
model on Honey-Data-15M. Experiments show that Bee-8B establishes a new
state-of-the-art (SOTA) for fully open MLLMs, achieving performance that is
competitive with, and in some cases surpasses, recent semi-open models such as
InternVL3.5-8B. Our work delivers to the community a suite of foundational
resources, including: the Honey-Data-15M corpus; the full-stack suite
comprising HoneyPipe and DataStudio; training recipes; an evaluation harness;
and the model weights. This effort demonstrates that a principled focus on data
quality is a key pathway to developing fully open MLLMs that are highly
competitive with their semi-open counterparts.
68f05e7636f8b025381e19af
Yi Zhang
claimed_verified
2025-10-16T06:35:15.644Z
63b2efb5922f26a27e76381c
https://cdn-avatars.huggingface.co/v1/production/uploads/63b2efb5922f26a27e76381c/zOQAt_xywiY8eTvvQOrmQ.png
Yi Zhang
user
uyzhang
68f05e7636f8b025381e19b0
Bolin Ni
68f05e7636f8b025381e19b1
Xin-Sheng Chen
68f05e7636f8b025381e19b2
Heng-Rui Zhang
68f05e7636f8b025381e19b3
Yongming Rao
68f05e7636f8b025381e19b4
Houwen Peng
68f05e7636f8b025381e19b5
Qinglin Lu
68f05e7636f8b025381e19b6
Han Hu
68f05e7636f8b025381e19b7
Meng-Hao Guo
claimed_verified
2025-10-16T06:38:31.512Z
62145614b670cb63a38075ba
/avatars/5e33debde75ae6c87640f63c48c560c6.svg
MenghaoGuo
user
MenghaoGuo
68f05e7636f8b025381e19b8
Shi-Min Hu
Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully
  Open MLLMs
Fully open multimodal large language models (MLLMs) currently lag behind
proprietary counterparts, primarily due to a significant gap in data quality
for supervised fine-tuning (SFT). Existing open-source datasets are often
plagued by widespread noise and a critical deficit in complex reasoning data,
such as Chain-of-Thought (CoT), which hinders the development of advanced model
capabilities. Addressing these challenges, our work makes three primary
contributions. First, we introduce Honey-Data-15M, a new SFT dataset comprising
approximately 15 million QA pairs, processed through multiple cleaning
techniques and enhanced with a novel dual-level (short and long) CoT enrichment
strategy. Second, we introduce HoneyPipe, the data curation pipeline, and its
underlying framework DataStudio, providing the community with a transparent and
adaptable methodology for data curation that moves beyond static dataset
releases. Finally, to validate our dataset and pipeline, we train Bee-8B, an 8B
model on Honey-Data-15M. Experiments show that Bee-8B establishes a new
state-of-the-art (SOTA) for fully open MLLMs, achieving performance that is
competitive with, and in some cases surpasses, recent semi-open models such as
InternVL3.5-8B. Our work delivers to the community a suite of foundational
resources, including: the Honey-Data-15M corpus; the full-stack suite
comprising HoneyPipe and DataStudio; training recipes; an evaluation harness;
and the model weights. This effort demonstrates that a principled focus on data
quality is a key pathway to developing fully open MLLMs that are highly
competitive with their semi-open counterparts.
A new dataset and pipeline for data curation improve the performance of fully open multimodal large language models, achieving state-of-the-art results competitive with semi-open models.
multimodal large language models supervised fine-tuning Chain-of-Thought Honey-Data-15M HoneyPipe DataStudio Bee-8B state-of-the-art