ColorFlow: Retrieval-Augmented Image Sequence Colorization
Automatic black-and-white image sequence colorization while preserving
character and object identity (ID) is a complex task with significant market
demand, such as in cartoon or comic series colorization. Despite advancements
in visual colorization using large-scale generative models like diffusion
models, challenges with controllability and identity consistency persist,
making current solutions unsuitable for industrial application.To address this,
we propose ColorFlow, a three-stage diffusion-based framework tailored for
image sequence colorization in industrial applications. Unlike existing methods
that require per-ID finetuning or explicit ID embedding extraction, we propose
a novel robust and generalizable Retrieval Augmented Colorization pipeline for
colorizing images with relevant color references. Our pipeline also features a
dual-branch design: one branch for color identity extraction and the other for
colorization, leveraging the strengths of diffusion models. We utilize the
self-attention mechanism in diffusion models for strong in-context learning and
color identity matching. To evaluate our model, we introduce ColorFlow-Bench, a
comprehensive benchmark for reference-based colorization. Results show that
ColorFlow outperforms existing models across multiple metrics, setting a new
standard in sequential image colorization and potentially benefiting the art
industry. We release our codes and models on our project page:
https://zhuang2002.github.io/ColorFlow/.
6760ea776fc1fd2b8ed9c802
Junhao Zhuang
claimed_verified
2024-12-17T08:03:56.429Z
64970d3d9c3b29dca8633f87
/avatars/11e3c9c66d28490d6d09925f9aa47cd1.svg
JunhaoZhuang
user
JunhaoZhuang
6760ea776fc1fd2b8ed9c803
Xuan Ju
admin_assigned
2024-12-17T09:25:47.793Z
62d4577bc85b0fcf7fde39bb
/avatars/a3a5729e33ae89ce9ba408830db3c835.svg
Xuan Ju
user
juxuan27
6760ea776fc1fd2b8ed9c804
Zhaoyang Zhang
claimed_verified
2024-12-17T08:03:54.477Z
658409ceca19ccf6d9989add
/avatars/3ac1dbd25e52435185babdeb3da28875.svg
Zhaoyang Zhang
user
ZyZcuhk
6760ea776fc1fd2b8ed9c805
Yong Liu
6760ea776fc1fd2b8ed9c806
Shiyi Zhang
claimed_verified
2025-05-09T08:55:21.766Z
6315d306a9456afe2b9bf34a
/avatars/7285b4e7d84b528d1a50f8ee4eb10727.svg
ElevenZ
user
shiyi0408
6760ea776fc1fd2b8ed9c807
Chun Yuan
6760ea776fc1fd2b8ed9c808
Ying Shan
admin_assigned
2024-12-17T09:25:16.323Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
ColorFlow: Retrieval-Augmented Image Sequence Colorization
Automatic black-and-white image sequence colorization while preserving
character and object identity (ID) is a complex task with significant market
demand, such as in cartoon or comic series colorization. Despite advancements
in visual colorization using large-scale generative models like diffusion
models, challenges with controllability and identity consistency persist,
making current solutions unsuitable for industrial application.To address this,
we propose ColorFlow, a three-stage diffusion-based framework tailored for
image sequence colorization in industrial applications. Unlike existing methods
that require per-ID finetuning or explicit ID embedding extraction, we propose
a novel robust and generalizable Retrieval Augmented Colorization pipeline for
colorizing images with relevant color references. Our pipeline also features a
dual-branch design: one branch for color identity extraction and the other for
colorization, leveraging the strengths of diffusion models. We utilize the
self-attention mechanism in diffusion models for strong in-context learning and
color identity matching. To evaluate our model, we introduce ColorFlow-Bench, a
comprehensive benchmark for reference-based colorization. Results show that
ColorFlow outperforms existing models across multiple metrics, setting a new
standard in sequential image colorization and potentially benefiting the art
industry. We release our codes and models on our project page:
https://zhuang2002.github.io/ColorFlow/.
ColorFlow, a three-stage diffusion-based framework with a dual-branch design, achieves superior image sequence colorization by incorporating retrieval augmented colorization and self-attention mechanisms, outperforming existing models.
diffusion-based framework retrieval augmented colorization dual-branch design self-attention mechanism reference-based colorization