PIXART-δ: Fast and Controllable Image Generation with Latent
  Consistency Models
This technical report introduces PIXART-{\delta}, a text-to-image synthesis
framework that integrates the Latent Consistency Model (LCM) and ControlNet
into the advanced PIXART-{\alpha} model. PIXART-{\alpha} is recognized for its
ability to generate high-quality images of 1024px resolution through a
remarkably efficient training process. The integration of LCM in
PIXART-{\delta} significantly accelerates the inference speed, enabling the
production of high-quality images in just 2-4 steps. Notably, PIXART-{\delta}
achieves a breakthrough 0.5 seconds for generating 1024x1024 pixel images,
marking a 7x improvement over the PIXART-{\alpha}. Additionally,
PIXART-{\delta} is designed to be efficiently trainable on 32GB V100 GPUs
within a single day. With its 8-bit inference capability (von Platen et al.,
2023), PIXART-{\delta} can synthesize 1024px images within 8GB GPU memory
constraints, greatly enhancing its usability and accessibility. Furthermore,
incorporating a ControlNet-like module enables fine-grained control over
text-to-image diffusion models. We introduce a novel ControlNet-Transformer
architecture, specifically tailored for Transformers, achieving explicit
controllability alongside high-quality image generation. As a state-of-the-art,
open-source image generation model, PIXART-{\delta} offers a promising
alternative to the Stable Diffusion family of models, contributing
significantly to text-to-image synthesis.
659f529fd56cbade8db370aa
Junsong Chen
admin_assigned
2024-01-11T10:37:30.196Z
645b5b09bc7518912e1f9733
/avatars/4d35f728b41f93881a9b67c337f4d1df.svg
Chen
user
Lawrence-cj
659f529fd56cbade8db370ab
Yue Wu
659f529fd56cbade8db370ac
Simian Luo
admin_assigned
2024-01-11T10:33:40.265Z
652a4f174d0eef4a2bf85096
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ofYN04WWtD60oeDP45pQN.jpeg
Simian Luo
user
SimianLuo
659f529fd56cbade8db370ad
Enze Xie
admin_assigned
2024-01-11T10:38:55.039Z
616961bdd3f656f79ad18ec1
https://cdn-avatars.huggingface.co/v1/production/uploads/616961bdd3f656f79ad18ec1/NYpxas378lDevRuUDepJJ.jpeg
xieenze
user
xieenze
659f529fd56cbade8db370ae
Sayak Paul
claimed_verified
2024-01-11T08:13:18.092Z
5f7fbd813e94f16a85448745
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
user
sayakpaul
659f529fd56cbade8db370af
Ping Luo
659f529fd56cbade8db370b0
Hang Zhao
admin_assigned
2024-01-11T10:35:02.476Z
63bc8467b8c61b8aa49c3e11
/avatars/f75a33b6bba1bb6aecbe018dcef39131.svg
Hang Zhao
user
sgzhaohang
659f529fd56cbade8db370b1
Zhenguo Li
admin_assigned
2024-01-11T10:35:30.043Z
639944bef7c4648254d124f0
/avatars/c5cbb87cd51cc1341844f67d42c55151.svg
lzg
user
lizhenguo
PIXART-δ: Fast and Controllable Image Generation with Latent
  Consistency Models
This technical report introduces PIXART-{\delta}, a text-to-image synthesis
framework that integrates the Latent Consistency Model (LCM) and ControlNet
into the advanced PIXART-{\alpha} model. PIXART-{\alpha} is recognized for its
ability to generate high-quality images of 1024px resolution through a
remarkably efficient training process. The integration of LCM in
PIXART-{\delta} significantly accelerates the inference speed, enabling the
production of high-quality images in just 2-4 steps. Notably, PIXART-{\delta}
achieves a breakthrough 0.5 seconds for generating 1024x1024 pixel images,
marking a 7x improvement over the PIXART-{\alpha}. Additionally,
PIXART-{\delta} is designed to be efficiently trainable on 32GB V100 GPUs
within a single day. With its 8-bit inference capability (von Platen et al.,
2023), PIXART-{\delta} can synthesize 1024px images within 8GB GPU memory
constraints, greatly enhancing its usability and accessibility. Furthermore,
incorporating a ControlNet-like module enables fine-grained control over
text-to-image diffusion models. We introduce a novel ControlNet-Transformer
architecture, specifically tailored for Transformers, achieving explicit
controllability alongside high-quality image generation. As a state-of-the-art,
open-source image generation model, PIXART-{\delta} offers a promising
alternative to the Stable Diffusion family of models, contributing
significantly to text-to-image synthesis.
PIXART-{\delta} integrates Latent Consistency Model (LCM) and ControlNet into PIXART-{\alpha} for accelerated, high-quality text-to-image synthesis with efficient training and inference.
Latent Consistency Model (LCM) ControlNet text-to-image synthesis diffusion models ControlNet-Transformer 8-bit inference