DELIFT: Data Efficient Language model Instruction Fine Tuning
Fine-tuning large language models (LLMs) is essential for enhancing their
performance on specific tasks but is often resource-intensive due to redundant
or uninformative data. To address this inefficiency, we introduce DELIFT (Data
Efficient Language model Instruction Fine-Tuning), a novel algorithm that
systematically optimizes data selection across the three key stages of
fine-tuning: (1) instruction tuning, (2) task-specific fine-tuning (e.g.,
reasoning, question-answering), and (3) continual fine-tuning (e.g.,
incorporating new data versions). Unlike existing methods that focus on
single-stage optimization or rely on computationally intensive gradient
calculations, DELIFT operates efficiently across all stages. Central to our
approach is a pairwise utility metric that quantifies how beneficial a data
sample is for improving the model's responses to other samples, effectively
measuring the informational value relative to the model's current capabilities.
By leveraging different submodular functions applied to this metric, DELIFT
selects diverse and optimal subsets that are useful across all stages of
fine-tuning. Experiments across various tasks and model scales demonstrate that
DELIFT can reduce the fine-tuning data size by up to 70% without compromising
performance, offering significant computational savings and outperforming
existing methods in both efficiency and efficacy.
67319dedb26b12f7069a16f4
Ishika Agarwal
extracted_confirmed
2025-02-18T18:06:35.675Z
6391e4e984afa726d66180b9
/avatars/e437e2820745b522a868b8da27d9a11f.svg
Ishika Agarwal
user
ishikaa
67319dedb26b12f7069a16f5
Krishna Killamsetty
claimed_verified
2025-04-17T08:06:17.827Z
64e4f834a07739bcc4906a3f
/avatars/19df353130dfdfb826ec0aac28904d5c.svg
Krishnateja Killamsetty
user
krishnatejak
67319dedb26b12f7069a16f6
Lucian Popa
67319dedb26b12f7069a16f7
Marina Danilevksy
DELIFT: Data Efficient Language model Instruction Fine Tuning
Fine-tuning large language models (LLMs) is essential for enhancing their
performance on specific tasks but is often resource-intensive due to redundant
or uninformative data. To address this inefficiency, we introduce DELIFT (Data
Efficient Language model Instruction Fine-Tuning), a novel algorithm that
systematically optimizes data selection across the three key stages of
fine-tuning: (1) instruction tuning, (2) task-specific fine-tuning (e.g.,
reasoning, question-answering), and (3) continual fine-tuning (e.g.,
incorporating new data versions). Unlike existing methods that focus on
single-stage optimization or rely on computationally intensive gradient
calculations, DELIFT operates efficiently across all stages. Central to our
approach is a pairwise utility metric that quantifies how beneficial a data
sample is for improving the model's responses to other samples, effectively
measuring the informational value relative to the model's current capabilities.
By leveraging different submodular functions applied to this metric, DELIFT
selects diverse and optimal subsets that are useful across all stages of
fine-tuning. Experiments across various tasks and model scales demonstrate that
DELIFT can reduce the fine-tuning data size by up to 70% without compromising
performance, offering significant computational savings and outperforming
existing methods in both efficiency and efficacy.
DELIFT optimizes data selection during LLM fine-tuning across multiple stages using a pairwise utility metric, reducing data size and computational costs while maintaining high performance.
DELIFT large language models fine-tuning instruction tuning task-specific fine-tuning continual fine-tuning pairwise utility metric informational value submodular functions data efficiency computational savings