DMM: Building a Versatile Image Generation Model via Distillation-Based
  Model Merging
The success of text-to-image (T2I) generation models has spurred a
proliferation of numerous model checkpoints fine-tuned from the same base model
on various specialized datasets. This overwhelming specialized model production
introduces new challenges for high parameter redundancy and huge storage cost,
thereby necessitating the development of effective methods to consolidate and
unify the capabilities of diverse powerful models into a single one. A common
practice in model merging adopts static linear interpolation in the parameter
space to achieve the goal of style mixing. However, it neglects the features of
T2I generation task that numerous distinct models cover sundry styles which may
lead to incompatibility and confusion in the merged model. To address this
issue, we introduce a style-promptable image generation pipeline which can
accurately generate arbitrary-style images under the control of style vectors.
Based on this design, we propose the score distillation based model merging
paradigm (DMM), compressing multiple models into a single versatile T2I model.
Moreover, we rethink and reformulate the model merging task in the context of
T2I generation, by presenting new merging goals and evaluation protocols. Our
experiments demonstrate that DMM can compactly reorganize the knowledge from
multiple teacher models and achieve controllable arbitrary-style generation.
6801d913a0cf74448f93d5c8
Tianhui Song
claimed_verified
2025-04-18T09:38:44.267Z
649e7693a83143427691769c
/avatars/d04f7b3d417423abaa053375212da21f.svg
Tianhui Song
user
sthuihui
6801d913a0cf74448f93d5c9
Weixin Feng
6801d913a0cf74448f93d5ca
Shuai Wang
claimed_verified
2025-04-18T09:38:42.164Z
66615c855fd9d736e670e0a9
/avatars/0ff3127b513552432a7c651e21d7f283.svg
wangshuai
user
wangsssssss
6801d913a0cf74448f93d5cb
Xubin Li
6801d913a0cf74448f93d5cc
Tiezheng Ge
6801d913a0cf74448f93d5cd
Bo Zheng
6801d913a0cf74448f93d5ce
Limin Wang
DMM: Building a Versatile Image Generation Model via Distillation-Based
  Model Merging
The success of text-to-image (T2I) generation models has spurred a
proliferation of numerous model checkpoints fine-tuned from the same base model
on various specialized datasets. This overwhelming specialized model production
introduces new challenges for high parameter redundancy and huge storage cost,
thereby necessitating the development of effective methods to consolidate and
unify the capabilities of diverse powerful models into a single one. A common
practice in model merging adopts static linear interpolation in the parameter
space to achieve the goal of style mixing. However, it neglects the features of
T2I generation task that numerous distinct models cover sundry styles which may
lead to incompatibility and confusion in the merged model. To address this
issue, we introduce a style-promptable image generation pipeline which can
accurately generate arbitrary-style images under the control of style vectors.
Based on this design, we propose the score distillation based model merging
paradigm (DMM), compressing multiple models into a single versatile T2I model.
Moreover, we rethink and reformulate the model merging task in the context of
T2I generation, by presenting new merging goals and evaluation protocols. Our
experiments demonstrate that DMM can compactly reorganize the knowledge from
multiple teacher models and achieve controllable arbitrary-style generation.
The paper introduces score distillation based model merging for text-to-image generation, enabling a single versatile model to produce images in different styles based on style vectors.
text-to-image (T2I) parameter redundancy storage cost model merging static linear interpolation style vectors score distillation based model merging (DMM) style-promptable image generation