Control LLM: Controlled Evolution for Intelligence Retention in LLM
Large Language Models (LLMs) demand significant computational resources,
making it essential to enhance their capabilities without retraining from
scratch. A key challenge in this domain is catastrophic forgetting
(CF), which hampers performance during Continuous Pre-training (CPT) and
Continuous Supervised Fine-Tuning (CSFT). We propose Control LLM, a
novel approach that leverages parallel pre-trained and expanded transformer
blocks, aligning their hidden-states through interpolation strategies This
method effectively preserves performance on existing tasks while seamlessly
integrating new knowledge.
  Extensive experiments demonstrate the effectiveness of Control LLM in both
CPT and CSFT. On Llama3.1-8B-Instruct, it achieves significant improvements in
mathematical reasoning (+14.4% on Math-Hard) and coding performance (+10%
on MBPP-PLUS). On Llama3.1-8B, it enhances multilingual capabilities (+10.6%
on C-Eval, +6.8% on CMMLU, and +30.2% on CMMLU-0shot-CoT). It surpasses
existing methods and achieves SOTA among open-source models tuned from the same
base model, using substantially less data and compute. Crucially, these gains
are realized while preserving strong original capabilities, with minimal
degradation (<4.3% on MMLU) compared to >35% in open-source Math
and Coding models. This approach has been successfully deployed in LinkedIn's
GenAI-powered job seeker and Ads unit products.
  To support further research, we release the training and evaluation code
(https://github.com/linkedin/ControlLLM) along with models trained on
public datasets ( https://huggingface.co/ControlLLM) to the community.
6792cf69627180db59a51b7c
Haichao Wei
claimed_verified
2025-01-24T09:08:11.344Z
6199b1d090692f0d92388473
/avatars/72693e4d37ded1542cd2564879fd9a61.svg
Haichao Wei
user
hawei
6792cf69627180db59a51b7d
Yunxiang Ren
6792cf69627180db59a51b7e
Zhoutong Fu
6792cf69627180db59a51b7f
Aman Lunia
6792cf69627180db59a51b80
Yi-Lin Chen
6792cf69627180db59a51b81
Alice Leung
6792cf69627180db59a51b82
Ya Xu
Control LLM: Controlled Evolution for Intelligence Retention in LLM
Large Language Models (LLMs) demand significant computational resources,
making it essential to enhance their capabilities without retraining from
scratch. A key challenge in this domain is catastrophic forgetting
(CF), which hampers performance during Continuous Pre-training (CPT) and
Continuous Supervised Fine-Tuning (CSFT). We propose Control LLM, a
novel approach that leverages parallel pre-trained and expanded transformer
blocks, aligning their hidden-states through interpolation strategies This
method effectively preserves performance on existing tasks while seamlessly
integrating new knowledge.
  Extensive experiments demonstrate the effectiveness of Control LLM in both
CPT and CSFT. On Llama3.1-8B-Instruct, it achieves significant improvements in
mathematical reasoning (+14.4% on Math-Hard) and coding performance (+10%
on MBPP-PLUS). On Llama3.1-8B, it enhances multilingual capabilities (+10.6%
on C-Eval, +6.8% on CMMLU, and +30.2% on CMMLU-0shot-CoT). It surpasses
existing methods and achieves SOTA among open-source models tuned from the same
base model, using substantially less data and compute. Crucially, these gains
are realized while preserving strong original capabilities, with minimal
degradation (<4.3% on MMLU) compared to >35% in open-source Math
and Coding models. This approach has been successfully deployed in LinkedIn's
GenAI-powered job seeker and Ads unit products.
  To support further research, we release the training and evaluation code
(https://github.com/linkedin/ControlLLM) along with models trained on
public datasets ( https://huggingface.co/ControlLLM) to the community.
Control LLM enhances Large Language Models through parallel transformer blocks and interpolation, improving performance in continuous training and fine-tuning while minimizing catastrophic forgetting.
Large Language Models LLMs catastrophic forgetting Continuous Pre-training CPT Continuous Supervised Fine-Tuning CSFT Control LLM transformer blocks interpolation mathematical reasoning coding performance multilingual capabilities C-Eval CMMLU CMMLU-0shot-CoT GenAI MMLU