Enabling Flexible Multi-LLM Integration for Scalable Knowledge
  Aggregation
Large language models (LLMs) have shown remarkable promise but remain
challenging to continually improve through traditional finetuning, particularly
when integrating capabilities from other specialized LLMs. Popular methods like
ensemble and weight merging require substantial memory and struggle to adapt to
changing data environments. Recent efforts have transferred knowledge from
multiple LLMs into a single target model; however, they suffer from
interference and degraded performance among tasks, largely due to limited
flexibility in candidate selection and training pipelines. To address these
issues, we propose a framework that adaptively selects and aggregates knowledge
from diverse LLMs to build a single, stronger model, avoiding the high memory
overhead of ensemble and inflexible weight merging. Specifically, we design an
adaptive selection network that identifies the most relevant source LLMs based
on their scores, thereby reducing knowledge interference. We further propose a
dynamic weighted fusion strategy that accounts for the inherent strengths of
candidate LLMs, along with a feedback-driven loss function that prevents the
selector from converging on a single subset of sources. Experimental results
demonstrate that our method can enable a more stable and scalable knowledge
aggregation process while reducing knowledge interference by up to 50% compared
to existing approaches. Code is avaliable at
https://github.com/ZLKong/LLM_Integration
683d0ac47852d920b7dc3dc5
Zhenglun Kong
683d0ac47852d920b7dc3dc6
Zheng Zhan
683d0ac47852d920b7dc3dc7
Shiyue Hou
683d0ac47852d920b7dc3dc8
Yifan Gong
683d0ac47852d920b7dc3dc9
Xin Meng
683d0ac47852d920b7dc3dca
Pengwei Sui
683d0ac47852d920b7dc3dcb
Peiyan Dong
683d0ac47852d920b7dc3dcc
Xuan Shen
683d0ac47852d920b7dc3dcd
Zifeng Wang
683d0ac47852d920b7dc3dce
Pu Zhao
683d0ac47852d920b7dc3dcf
Hao Tang
683d0ac47852d920b7dc3dd0
Stratis Ioannidis
683d0ac47852d920b7dc3dd1
Yanzhi Wang
Enabling Flexible Multi-LLM Integration for Scalable Knowledge
  Aggregation
Large language models (LLMs) have shown remarkable promise but remain
challenging to continually improve through traditional finetuning, particularly
when integrating capabilities from other specialized LLMs. Popular methods like
ensemble and weight merging require substantial memory and struggle to adapt to
changing data environments. Recent efforts have transferred knowledge from
multiple LLMs into a single target model; however, they suffer from
interference and degraded performance among tasks, largely due to limited
flexibility in candidate selection and training pipelines. To address these
issues, we propose a framework that adaptively selects and aggregates knowledge
from diverse LLMs to build a single, stronger model, avoiding the high memory
overhead of ensemble and inflexible weight merging. Specifically, we design an
adaptive selection network that identifies the most relevant source LLMs based
on their scores, thereby reducing knowledge interference. We further propose a
dynamic weighted fusion strategy that accounts for the inherent strengths of
candidate LLMs, along with a feedback-driven loss function that prevents the
selector from converging on a single subset of sources. Experimental results
demonstrate that our method can enable a more stable and scalable knowledge
aggregation process while reducing knowledge interference by up to 50% compared
to existing approaches. Code is avaliable at
https://github.com/ZLKong/LLM_Integration
A framework for adaptive selection and dynamic weighted fusion of knowledge from multiple LLMs reduces interference and improves scalability in knowledge aggregation.
large language models fine-tuning ensemble weight merging adaptive selection network dynamic weighted fusion feedback-driven loss function knowledge interference