GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via
  General Samples Replay
The continual learning capability of large language models (LLMs) is crucial
for advancing artificial general intelligence. However, continual fine-tuning
LLMs across various domains often suffers from catastrophic forgetting,
characterized by: 1) significant forgetting of their general capabilities, and
2) sharp performance declines in previously learned tasks. To simultaneously
address both issues in a simple yet stable manner, we propose General Sample
Replay (GeRe), a framework that use usual pretraining texts for efficient
anti-forgetting. Beyond revisiting the most prevalent replay-based practices
under GeRe, we further leverage neural states to introduce a enhanced
activation states constrained optimization method using threshold-based margin
(TM) loss, which maintains activation state consistency during replay learning.
We are the first to validate that a small, fixed set of pre-collected general
replay samples is sufficient to resolve both concerns--retaining general
capabilities while promoting overall performance across sequential tasks.
Indeed, the former can inherently facilitate the latter. Through controlled
experiments, we systematically compare TM with different replay strategies
under the GeRe framework, including vanilla label fitting, logit imitation via
KL divergence and feature imitation via L1/L2 losses. Results demonstrate that
TM consistently improves performance and exhibits better robustness. Our work
paves the way for efficient replay of LLMs for the future. Our code and data
are available at https://github.com/Qznan/GeRe.
689c4728fab6fdd2e52ac951
Yunan Zhang
claimed_verified
2025-08-14T13:38:40.630Z
641d3e9a353524fe41f203e8
https://cdn-avatars.huggingface.co/v1/production/uploads/641d3e9a353524fe41f203e8/hxdQCUktxW-wp57c4VjeH.png
YonasZhang
user
Qznan
689c4728fab6fdd2e52ac952
Shuoran Jiang
689c4728fab6fdd2e52ac953
Mengchen Zhao
689c4728fab6fdd2e52ac954
Yuefeng Li
689c4728fab6fdd2e52ac955
Yang Fan
689c4728fab6fdd2e52ac956
Xiangping Wu
689c4728fab6fdd2e52ac957
Qingcai Chen
GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via
  General Samples Replay
The continual learning capability of large language models (LLMs) is crucial
for advancing artificial general intelligence. However, continual fine-tuning
LLMs across various domains often suffers from catastrophic forgetting,
characterized by: 1) significant forgetting of their general capabilities, and
2) sharp performance declines in previously learned tasks. To simultaneously
address both issues in a simple yet stable manner, we propose General Sample
Replay (GeRe), a framework that use usual pretraining texts for efficient
anti-forgetting. Beyond revisiting the most prevalent replay-based practices
under GeRe, we further leverage neural states to introduce a enhanced
activation states constrained optimization method using threshold-based margin
(TM) loss, which maintains activation state consistency during replay learning.
We are the first to validate that a small, fixed set of pre-collected general
replay samples is sufficient to resolve both concerns--retaining general
capabilities while promoting overall performance across sequential tasks.
Indeed, the former can inherently facilitate the latter. Through controlled
experiments, we systematically compare TM with different replay strategies
under the GeRe framework, including vanilla label fitting, logit imitation via
KL divergence and feature imitation via L1/L2 losses. Results demonstrate that
TM consistently improves performance and exhibits better robustness. Our work
paves the way for efficient replay of LLMs for the future. Our code and data
are available at https://github.com/Qznan/GeRe.
General Sample Replay (GeRe) framework with threshold-based margin (TM) loss addresses catastrophic forgetting in large language models by maintaining activation state consistency during replay learning.
continual learning large language models catastrophic forgetting General Sample Replay GeRe neural states enhanced activation states constrained optimization threshold-based margin TM loss replay learning vanilla label fitting logit imitation feature imitation