TransPixar: Advancing Text-to-Video Generation with Transparency
Text-to-video generative models have made significant strides, enabling
diverse applications in entertainment, advertising, and education. However,
generating RGBA video, which includes alpha channels for transparency, remains
a challenge due to limited datasets and the difficulty of adapting existing
models. Alpha channels are crucial for visual effects (VFX), allowing
transparent elements like smoke and reflections to blend seamlessly into
scenes. We introduce TransPixar, a method to extend pretrained video models for
RGBA generation while retaining the original RGB capabilities. TransPixar
leverages a diffusion transformer (DiT) architecture, incorporating
alpha-specific tokens and using LoRA-based fine-tuning to jointly generate RGB
and alpha channels with high consistency. By optimizing attention mechanisms,
TransPixar preserves the strengths of the original RGB model and achieves
strong alignment between RGB and alpha channels despite limited training data.
Our approach effectively generates diverse and consistent RGBA videos,
advancing the possibilities for VFX and interactive content creation.
677c92224a867d8292b0a4d6
Luozhou Wang
admin_assigned
2025-01-07T09:50:03.830Z
6340333a733f9eef46913dc8
https://cdn-avatars.huggingface.co/v1/production/uploads/6340333a733f9eef46913dc8/BwhKgkqbuuxWsJv2s_JEH.png
luozhou wang
user
wileewang
677c92224a867d8292b0a4d7
Yijun Li
677c92224a867d8292b0a4d8
Zhifei Chen
admin_assigned
2025-01-07T09:50:28.496Z
634a91c53a0cd2d4985dc98a
/avatars/fe680e683761c347cfa4bc9273b67b0d.svg
ZhiFei Chen
user
zhifeichen097
677c92224a867d8292b0a4d9
Jui-Hsien Wang
677c92224a867d8292b0a4da
Zhifei Zhang
677c92224a867d8292b0a4db
He Zhang
admin_assigned
2025-01-07T09:50:58.619Z
646c333136505117e22f08eb
/avatars/6b720565b3916a6f7d1bb7a73cd99a33.svg
He Zhang
user
HeZhang
677c92224a867d8292b0a4dc
Zhe Lin
677c92224a867d8292b0a4dd
Yingcong Chen
admin_assigned
2025-01-07T09:51:09.776Z
655cba1d87b67834000590e8
/avatars/3bd43b7c9351f65b8f38f4c8237a0146.svg
Yingcong Chen
user
yingcongchen
TransPixar: Advancing Text-to-Video Generation with Transparency
Text-to-video generative models have made significant strides, enabling
diverse applications in entertainment, advertising, and education. However,
generating RGBA video, which includes alpha channels for transparency, remains
a challenge due to limited datasets and the difficulty of adapting existing
models. Alpha channels are crucial for visual effects (VFX), allowing
transparent elements like smoke and reflections to blend seamlessly into
scenes. We introduce TransPixar, a method to extend pretrained video models for
RGBA generation while retaining the original RGB capabilities. TransPixar
leverages a diffusion transformer (DiT) architecture, incorporating
alpha-specific tokens and using LoRA-based fine-tuning to jointly generate RGB
and alpha channels with high consistency. By optimizing attention mechanisms,
TransPixar preserves the strengths of the original RGB model and achieves
strong alignment between RGB and alpha channels despite limited training data.
Our approach effectively generates diverse and consistent RGBA videos,
advancing the possibilities for VFX and interactive content creation.
TransPixar extends pretrained video models to generate RGBA videos using a diffusion transformer with LoRA-based fine-tuning, ensuring consistency between RGB and alpha channels.
diffusion transformer DiT alpha-specific tokens LoRA-based fine-tuning