GlyphControl: Glyph Conditional Control for Visual Text Generation
Recently, there has been a growing interest in developing diffusion-based
text-to-image generative models capable of generating coherent and well-formed
visual text. In this paper, we propose a novel and efficient approach called
GlyphControl to address this task. Unlike existing methods that rely on
character-aware text encoders like ByT5 and require retraining of text-to-image
models, our approach leverages additional glyph conditional information to
enhance the performance of the off-the-shelf Stable-Diffusion model in
generating accurate visual text. By incorporating glyph instructions, users can
customize the content, location, and size of the generated text according to
their specific requirements. To facilitate further research in visual text
generation, we construct a training benchmark dataset called LAION-Glyph. We
evaluate the effectiveness of our approach by measuring OCR-based metrics and
CLIP scores of the generated visual text. Our empirical evaluations demonstrate
that GlyphControl outperforms the recent DeepFloyd IF approach in terms of OCR
accuracy and CLIP scores, highlighting the efficacy of our method.
647568aca855203d8fef2728
Yukang Yang
647568aca855203d8fef2729
Dongnan Gui
647568aca855203d8fef272a
Yuhui Yuan
claimed_verified
2023-05-30T10:08:04.279Z
646f69a6041e48e1c4728de3
https://cdn-avatars.huggingface.co/v1/production/uploads/646f69a6041e48e1c4728de3/U5OaW6PgsXTXnfG03xs9Q.png
GlyphByT5
user
GlyphByT5
647568aca855203d8fef272b
Haisong Ding
647568aca855203d8fef272c
Han Hu
647568aca855203d8fef272d
Kai Chen
extracted_pending
2024-05-21T09:22:22.961Z
664c6083f48f9e269c593105
/avatars/8cd69f3811202b6e2a1cb13ee53a8148.svg
Kai Chen
user
chenkagi
GlyphControl: Glyph Conditional Control for Visual Text Generation
Recently, there has been a growing interest in developing diffusion-based
text-to-image generative models capable of generating coherent and well-formed
visual text. In this paper, we propose a novel and efficient approach called
GlyphControl to address this task. Unlike existing methods that rely on
character-aware text encoders like ByT5 and require retraining of text-to-image
models, our approach leverages additional glyph conditional information to
enhance the performance of the off-the-shelf Stable-Diffusion model in
generating accurate visual text. By incorporating glyph instructions, users can
customize the content, location, and size of the generated text according to
their specific requirements. To facilitate further research in visual text
generation, we construct a training benchmark dataset called LAION-Glyph. We
evaluate the effectiveness of our approach by measuring OCR-based metrics and
CLIP scores of the generated visual text. Our empirical evaluations demonstrate
that GlyphControl outperforms the recent DeepFloyd IF approach in terms of OCR
accuracy and CLIP scores, highlighting the efficacy of our method.
GlyphControl enhances text-to-image generation by using glyph instructions, outperforming DeepFloyd IF in OCR accuracy and CLIP scores, using Stable-Diffusion without retraining.
diffusion-based generative models GlyphControl character-aware text encoders ByT5 Stable-Diffusion glyph instructions LAION-Glyph dataset OCR CLIP scores DeepFloyd IF