MUSCLE: A Model Update Strategy for Compatible LLM Evolution
Large Language Models (LLMs) are frequently updated due to data or
architecture changes to improve their performance. When updating models,
developers often focus on increasing overall performance metrics with less
emphasis on being compatible with previous model versions. However, users often
build a mental model of the functionality and capabilities of a particular
machine learning model they are interacting with. They have to adapt their
mental model with every update -- a draining task that can lead to user
dissatisfaction. In practice, fine-tuned downstream task adapters rely on
pretrained LLM base models. When these base models are updated, these
user-facing downstream task models experience instance regression or negative
flips -- previously correct instances are now predicted incorrectly. This
happens even when the downstream task training procedures remain identical. Our
work aims to provide seamless model updates to a user in two ways. First, we
provide evaluation metrics for a notion of compatibility to prior model
versions, specifically for generative tasks but also applicable for
discriminative tasks. We observe regression and inconsistencies between
different model versions on a diverse set of tasks and model updates. Second,
we propose a training strategy to minimize the number of inconsistencies in
model updates, involving training of a compatibility model that can enhance
task fine-tuned language models. We reduce negative flips -- instances where a
prior model version was correct, but a new model incorrect -- by up to 40% from
Llama 1 to Llama 2.
66948ba1df5d51613344d461
Jessica Echterhoff
66948ba1df5d51613344d462
Fartash Faghri
66948ba1df5d51613344d463
Raviteja Vemulapalli
66948ba1df5d51613344d464
Ting-Yao Hu
66948ba1df5d51613344d465
Chun-Liang Li
66948ba1df5d51613344d466
Oncel Tuzel
66948ba1df5d51613344d467
Hadi Pouransari
extracted_pending
2024-07-15T02:38:26.354Z
653a8e65c75dc136bfb5b0f8
/avatars/ef252794236ce0fe2debf12773c95bb2.svg
Hadi Pouransari
user
hpouransari
MUSCLE: A Model Update Strategy for Compatible LLM Evolution
Large Language Models (LLMs) are frequently updated due to data or
architecture changes to improve their performance. When updating models,
developers often focus on increasing overall performance metrics with less
emphasis on being compatible with previous model versions. However, users often
build a mental model of the functionality and capabilities of a particular
machine learning model they are interacting with. They have to adapt their
mental model with every update -- a draining task that can lead to user
dissatisfaction. In practice, fine-tuned downstream task adapters rely on
pretrained LLM base models. When these base models are updated, these
user-facing downstream task models experience instance regression or negative
flips -- previously correct instances are now predicted incorrectly. This
happens even when the downstream task training procedures remain identical. Our
work aims to provide seamless model updates to a user in two ways. First, we
provide evaluation metrics for a notion of compatibility to prior model
versions, specifically for generative tasks but also applicable for
discriminative tasks. We observe regression and inconsistencies between
different model versions on a diverse set of tasks and model updates. Second,
we propose a training strategy to minimize the number of inconsistencies in
model updates, involving training of a compatibility model that can enhance
task fine-tuned language models. We reduce negative flips -- instances where a
prior model version was correct, but a new model incorrect -- by up to 40% from
Llama 1 to Llama 2.
The work provides evaluation metrics and a training strategy to minimize inconsistencies and negative flips in Large Language Model updates, ensuring better compatibility with prior model versions for downstream tasks.
Large Language Models (LLMs) generative tasks discriminative tasks instance regression negative flips compatibility model task fine-tuned language models