Montessori-Instruct: Generate Influential Training Data Tailored for
  Student Learning
Synthetic data has been widely used to train large language models, but their
generative nature inevitably introduces noisy, non-informative, and misleading
learning signals. In this paper, we propose Montessori-Instruct, a novel data
synthesis framework that tailors the data synthesis ability of the teacher
language model toward the student language model's learning process.
Specifically, we utilize local data influence of synthetic training data points
on students to characterize students' learning preferences. Then, we train the
teacher model with Direct Preference Optimization (DPO) to generate synthetic
data tailored toward student learning preferences. Experiments with
Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and
MT-Bench demonstrate that Montessori-Instruct significantly outperforms
standard synthesis methods by 18.35\% and 46.24\% relatively. Our method also
beats data synthesized by a stronger teacher model, GPT-4o. Further analysis
confirms the benefits of teacher's learning to generate more influential
training data in the student's improved learning, the advantages of local data
influence in accurately measuring student preferences, and the robustness of
Montessori-Instruct across different student models. Our code and data are
open-sourced at https://github.com/cxcscmu/Montessori-Instruct.
671664c4f262be795484c53c
Xiaochuan Li
claimed_verified
2025-05-20T07:23:33.008Z
64b103cf372d434077206750
/avatars/ba0eb4fc712a8b9b93ceb30d11859ec2.svg
Xiaochuan Li
user
lixiaochuan2020
671664c4f262be795484c53d
Zichun Yu
claimed_verified
2025-02-21T16:15:26.547Z
649a943f575e60d3a87cfcdf
/avatars/879f79628a30dddeab6e36e7d82bdbaa.svg
 Zichun Yu
user
yuzc19
671664c4f262be795484c53e
Chenyan Xiong
Montessori-Instruct: Generate Influential Training Data Tailored for
  Student Learning
Synthetic data has been widely used to train large language models, but their
generative nature inevitably introduces noisy, non-informative, and misleading
learning signals. In this paper, we propose Montessori-Instruct, a novel data
synthesis framework that tailors the data synthesis ability of the teacher
language model toward the student language model's learning process.
Specifically, we utilize local data influence of synthetic training data points
on students to characterize students' learning preferences. Then, we train the
teacher model with Direct Preference Optimization (DPO) to generate synthetic
data tailored toward student learning preferences. Experiments with
Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and
MT-Bench demonstrate that Montessori-Instruct significantly outperforms
standard synthesis methods by 18.35\% and 46.24\% relatively. Our method also
beats data synthesized by a stronger teacher model, GPT-4o. Further analysis
confirms the benefits of teacher's learning to generate more influential
training data in the student's improved learning, the advantages of local data
influence in accurately measuring student preferences, and the robustness of
Montessori-Instruct across different student models. Our code and data are
open-sourced at https://github.com/cxcscmu/Montessori-Instruct.
Montessori-Instruct enhances language model training by optimizing synthetic data generation to align with student learning preferences, outperforming standard methods and a more powerful teacher model.
local data influence Direct Preference Optimization (DPO) Alpaca Eval MT-Bench Llama3-8B-Instruct Llama3-8B GPT-4o