Your Student is Better Than Expected: Adaptive Teacher-Student
  Collaboration for Text-Conditional Diffusion Models
Knowledge distillation methods have recently shown to be a promising
direction to speedup the synthesis of large-scale diffusion models by requiring
only a few inference steps. While several powerful distillation methods were
recently proposed, the overall quality of student samples is typically lower
compared to the teacher ones, which hinders their practical usage. In this
work, we investigate the relative quality of samples produced by the teacher
text-to-image diffusion model and its distilled student version. As our main
empirical finding, we discover that a noticeable portion of student samples
exhibit superior fidelity compared to the teacher ones, despite the
``approximate'' nature of the student. Based on this finding, we propose an
adaptive collaboration between student and teacher diffusion models for
effective text-to-image synthesis. Specifically, the distilled model produces
the initial sample, and then an oracle decides whether it needs further
improvements with a slow teacher model. Extensive experiments demonstrate that
the designed pipeline surpasses state-of-the-art text-to-image alternatives for
various inference budgets in terms of human preference. Furthermore, the
proposed approach can be naturally used in popular applications such as
text-guided image editing and controllable generation.
658139c398aa9fcdd270d6d3
Nikita Starodubcev
admin_assigned
2023-12-19T10:12:46.126Z
6410d3a4cfbe9c4400233d1e
/avatars/9e1c4e0ea5f6964c90013f6e19c41db1.svg
nikita
user
quickjkee
658139c398aa9fcdd270d6d4
Artem Fedorov
admin_assigned
2023-12-19T10:11:51.167Z
6343fcfad236b24788be85f9
/avatars/72fe4c2b3aeb3e2ab87cdaf70b6615b8.svg
Artem Fedorov
user
Ok0lo
658139c398aa9fcdd270d6d5
Artem Babenko
658139c398aa9fcdd270d6d6
Dmitry Baranchuk
admin_assigned
2023-12-19T10:12:01.911Z
62b6cc49752323892323bc04
https://cdn-avatars.huggingface.co/v1/production/uploads/62b6cc49752323892323bc04/gGBld1KJIP9AIpd81L3PC.jpeg
Dmitry Baranchuk
user
dbaranchuk
Your Student is Better Than Expected: Adaptive Teacher-Student
  Collaboration for Text-Conditional Diffusion Models
Knowledge distillation methods have recently shown to be a promising
direction to speedup the synthesis of large-scale diffusion models by requiring
only a few inference steps. While several powerful distillation methods were
recently proposed, the overall quality of student samples is typically lower
compared to the teacher ones, which hinders their practical usage. In this
work, we investigate the relative quality of samples produced by the teacher
text-to-image diffusion model and its distilled student version. As our main
empirical finding, we discover that a noticeable portion of student samples
exhibit superior fidelity compared to the teacher ones, despite the
``approximate'' nature of the student. Based on this finding, we propose an
adaptive collaboration between student and teacher diffusion models for
effective text-to-image synthesis. Specifically, the distilled model produces
the initial sample, and then an oracle decides whether it needs further
improvements with a slow teacher model. Extensive experiments demonstrate that
the designed pipeline surpasses state-of-the-art text-to-image alternatives for
various inference budgets in terms of human preference. Furthermore, the
proposed approach can be naturally used in popular applications such as
text-guided image editing and controllable generation.
An adaptive collaboration between a distilled student model and a teacher model for text-to-image synthesis achieves superior results compared to state-of-the-art methods across different inference budgets.
knowledge distillation diffusion models text-to-image synthesis teacher model student model adaptive collaboration human preference text-guided image editing controllable generation