SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute
  in Linear Diffusion Transformer
This paper presents SANA-1.5, a linear Diffusion Transformer for efficient
scaling in text-to-image generation. Building upon SANA-1.0, we introduce three
key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that
enables scaling from 1.6B to 4.8B parameters with significantly reduced
computational resources, combined with a memory-efficient 8-bit optimizer. (2)
Model Depth Pruning: A block importance analysis technique for efficient model
compression to arbitrary sizes with minimal quality loss. (3) Inference-time
Scaling: A repeated sampling strategy that trades computation for model
capacity, enabling smaller models to match larger model quality at inference
time. Through these strategies, SANA-1.5 achieves a text-image alignment score
of 0.72 on GenEval, which can be further improved to 0.80 through inference
scaling, establishing a new SoTA on GenEval benchmark. These innovations enable
efficient model scaling across different compute budgets while maintaining high
quality, making high-quality image generation more accessible.
679d7fae1f4b90cfa7b74d0b
Enze Xie
679d7fae1f4b90cfa7b74d0c
Junsong Chen
claimed_verified
2025-03-20T10:54:32.699Z
645b5b09bc7518912e1f9733
/avatars/4d35f728b41f93881a9b67c337f4d1df.svg
Chen
user
Lawrence-cj
679d7fae1f4b90cfa7b74d0d
Yuyang Zhao
679d7fae1f4b90cfa7b74d0e
Jincheng Yu
679d7fae1f4b90cfa7b74d0f
Ligeng Zhu
679d7fae1f4b90cfa7b74d10
Yujun Lin
679d7fae1f4b90cfa7b74d11
Zhekai Zhang
679d7fae1f4b90cfa7b74d12
Muyang Li
679d7fae1f4b90cfa7b74d13
Junyu Chen
679d7fae1f4b90cfa7b74d14
Han Cai
679d7fae1f4b90cfa7b74d15
Bingchen Liu
679d7fae1f4b90cfa7b74d16
Daquan Zhou
679d7fae1f4b90cfa7b74d17
Song Han
SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute
  in Linear Diffusion Transformer
This paper presents SANA-1.5, a linear Diffusion Transformer for efficient
scaling in text-to-image generation. Building upon SANA-1.0, we introduce three
key innovations: (1) Efficient Training Scaling: A depth-growth paradigm that
enables scaling from 1.6B to 4.8B parameters with significantly reduced
computational resources, combined with a memory-efficient 8-bit optimizer. (2)
Model Depth Pruning: A block importance analysis technique for efficient model
compression to arbitrary sizes with minimal quality loss. (3) Inference-time
Scaling: A repeated sampling strategy that trades computation for model
capacity, enabling smaller models to match larger model quality at inference
time. Through these strategies, SANA-1.5 achieves a text-image alignment score
of 0.72 on GenEval, which can be further improved to 0.80 through inference
scaling, establishing a new SoTA on GenEval benchmark. These innovations enable
efficient model scaling across different compute budgets while maintaining high
quality, making high-quality image generation more accessible.
Diffusion Transformer depth-growth paradigm memory-efficient 8-bit optimizer block importance analysis model depth pruning repeated sampling strategy text-image alignment score GenEval SoTA