Evolutionary Optimization of Model Merging Recipes
We present a novel application of evolutionary algorithms to automate the
creation of powerful foundation models. While model merging has emerged as a
promising approach for LLM development due to its cost-effectiveness, it
currently relies on human intuition and domain knowledge, limiting its
potential. Here, we propose an evolutionary approach that overcomes this
limitation by automatically discovering effective combinations of diverse
open-source models, harnessing their collective intelligence without requiring
extensive additional training data or compute. Our approach operates in both
parameter space and data flow space, allowing for optimization beyond just the
weights of the individual models. This approach even facilitates cross-domain
merging, generating models like a Japanese LLM with Math reasoning
capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art
performance on a variety of established Japanese LLM benchmarks, even
surpassing models with significantly more parameters, despite not being
explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM
generated through our approach demonstrates its effectiveness in describing
Japanese culture-specific content, outperforming previous Japanese VLMs. This
work not only contributes new state-of-the-art models back to the open-source
community, but also introduces a new paradigm for automated model composition,
paving the way for exploring alternative, efficient approaches to foundation
model development.
65fb7f71ec1f99ad1f01f4ea
Takuya Akiba
admin_assigned
2024-03-21T09:07:28.747Z
6482810dba6c556892f6f257
https://cdn-avatars.huggingface.co/v1/production/uploads/6482810dba6c556892f6f257/c7-wiVKenXiRtwnRpnjZN.jpeg
Takuya Akiba
user
iwiwi
65fb7f71ec1f99ad1f01f4eb
Makoto Shing
claimed_verified
2024-03-21T08:26:55.128Z
60c2e7747a42b2edc5d2ccf7
https://cdn-avatars.huggingface.co/v1/production/uploads/60c2e7747a42b2edc5d2ccf7/FGlJt3Zd6NEcE-DKC3-dL.jpeg
Makoto Shing
user
mkshing
65fb7f71ec1f99ad1f01f4ec
Yujin Tang
admin_assigned
2024-03-21T09:07:55.475Z
63285beda4102056bc198395
/avatars/6c936818a3814c1c32a8eaae8b8cb6d6.svg
yujin tang
user
tyj2022
65fb7f71ec1f99ad1f01f4ed
Qi Sun
claimed_verified
2024-03-21T10:02:42.615Z
644b983f0fbe4830f192c4f5
https://cdn-avatars.huggingface.co/v1/production/uploads/644b983f0fbe4830f192c4f5/4eX4OS0gnXGqLu3eaMml_.png
Qi Sun
user
lfsm
65fb7f71ec1f99ad1f01f4ee
David Ha
admin_assigned
2024-03-21T09:08:27.543Z
6335ea47b5a0e02f80538bf3
/avatars/fce4e08ae86339952e35ff73680da7e9.svg
David
user
davidha
Evolutionary Optimization of Model Merging Recipes
We present a novel application of evolutionary algorithms to automate the
creation of powerful foundation models. While model merging has emerged as a
promising approach for LLM development due to its cost-effectiveness, it
currently relies on human intuition and domain knowledge, limiting its
potential. Here, we propose an evolutionary approach that overcomes this
limitation by automatically discovering effective combinations of diverse
open-source models, harnessing their collective intelligence without requiring
extensive additional training data or compute. Our approach operates in both
parameter space and data flow space, allowing for optimization beyond just the
weights of the individual models. This approach even facilitates cross-domain
merging, generating models like a Japanese LLM with Math reasoning
capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art
performance on a variety of established Japanese LLM benchmarks, even
surpassing models with significantly more parameters, despite not being
explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM
generated through our approach demonstrates its effectiveness in describing
Japanese culture-specific content, outperforming previous Japanese VLMs. This
work not only contributes new state-of-the-art models back to the open-source
community, but also introduces a new paradigm for automated model composition,
paving the way for exploring alternative, efficient approaches to foundation
model development.
An evolutionary algorithm automates model merging, discovering effective combinations of open-source models without additional training data or compute, achieving state-of-the-art performance on benchmarks and culturally-aware tasks.
evolutionary algorithms foundation models model merging LLM parameter space data flow space cross-domain merging Japanese LLM Math reasoning Japanese VLM