Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance
Aladin Djuhera, Swanand Kadhe, Syed Zawad, Farhan Ahmed, Heiko Ludwig, Holger Boche
Recent work on large language models (LLMs) has increasingly focused on post-training and alignment with datasets curated to enhance instruction following, world knowledge, and specialized skills. However, post-training datasets used by both open- and closed-source LLMs remain inaccessible to the public, with limited information about their construction process. This lack of transparency has motivated the development of open-source post-training corpora. While training on these datasets can yield performance comparable to that of leading models, their large size and complexity make systematic comparisons challenging, particularly under common compute and memory constraints. As a result, it is difficult to attribute performance gains to individual samples, task categories, or curation strategies when assessing data quality. In this work, we conduct a comprehensive analysis of two widely used post-training datasets, Tulu-3-SFT-Mix and SmolTalk, to reveal their structural and qualitative similarities and differences. Using the MagPie framework, we annotate every sample with detailed quality metrics, including turn structure (single-turn vs. multi-turn), task category, input quality, and response quality. Through controlled experiments, we investigate three key questions: (1) How many high-quality examples are needed to achieve stable performance gains? (2) What is the marginal utility of multi-turn dialogues compared to single-turn prompts? (3) How do different data mixture ratios influence model outcomes? Our findings offer actionable insights for designing effective post-training datasets that improve model performance within practical resource limits. To support future research, we publicly release our annotated versions of both datasets.