Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and
  Refinement
The quality of Supervised Fine-Tuning (SFT) data plays a critical role in
enhancing the conversational capabilities of Large Language Models (LLMs).
However, as LLMs become more advanced, the availability of high-quality
human-annotated SFT data has become a significant bottleneck, necessitating a
greater reliance on synthetic training data. In this work, we introduce Condor,
a novel two-stage synthetic data generation framework that incorporates World
Knowledge Tree and Self-Reflection Refinement to produce high-quality SFT data
at scale. Our experimental results demonstrate that a base model fine-tuned on
only 20K Condor-generated samples achieves superior performance compared to
counterparts. The additional refinement stage in Condor further enables
iterative self-improvement for LLMs at various scales (up to 72B), validating
the effectiveness of our approach. Furthermore, our investigation into the
scaling for synthetic data in post-training reveals substantial unexplored
potential for performance improvements, opening promising avenues for future
research.
67906c674932687e24e0cc08
Maosong Cao
67906c674932687e24e0cc09
Taolin Zhang
admin_assigned
2025-01-22T16:00:36.088Z
62a2b0f3575099d76bdc7259
https://cdn-avatars.huggingface.co/v1/production/uploads/1654829516458-62a2b0f3575099d76bdc7259.png
Taolin Zhang
user
ruxian
67906c674932687e24e0cc0a
Mo Li
admin_assigned
2025-01-22T16:01:30.729Z
674fe4239cad8e296c9f2d1e
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iZ22kVJClIzEK35oxzsp3.png
Mo Li
user
MoLi-DHU
67906c674932687e24e0cc0b
Chuyu Zhang
67906c674932687e24e0cc0c
Yunxin Liu
67906c674932687e24e0cc0d
Haodong Duan
admin_assigned
2025-01-22T16:01:03.878Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
67906c674932687e24e0cc0e
Songyang Zhang
claimed_verified
2025-01-22T10:08:23.652Z
630716d11801ecc7d2595021
/avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
Songyang Zhang
user
zsytony
67906c674932687e24e0cc0f
Kai Chen
Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and
  Refinement
The quality of Supervised Fine-Tuning (SFT) data plays a critical role in
enhancing the conversational capabilities of Large Language Models (LLMs).
However, as LLMs become more advanced, the availability of high-quality
human-annotated SFT data has become a significant bottleneck, necessitating a
greater reliance on synthetic training data. In this work, we introduce Condor,
a novel two-stage synthetic data generation framework that incorporates World
Knowledge Tree and Self-Reflection Refinement to produce high-quality SFT data
at scale. Our experimental results demonstrate that a base model fine-tuned on
only 20K Condor-generated samples achieves superior performance compared to
counterparts. The additional refinement stage in Condor further enables
iterative self-improvement for LLMs at various scales (up to 72B), validating
the effectiveness of our approach. Furthermore, our investigation into the
scaling for synthetic data in post-training reveals substantial unexplored
potential for performance improvements, opening promising avenues for future
research.
Condor, a two-stage synthetic data generation framework, enhances the performance of large language models through World Knowledge Tree and Self-Reflection Refinement, generating high-quality supervised fine-tuning data.
Supervised Fine-Tuning Large Language Models Condor World Knowledge Tree Self-Reflection Refinement