Omni-Effects: Unified and Spatially-Controllable Visual Effects
  Generation
Visual effects (VFX) are essential visual enhancements fundamental to modern
cinematic production. Although video generation models offer cost-efficient
solutions for VFX production, current methods are constrained by per-effect
LoRA training, which limits generation to single effects. This fundamental
limitation impedes applications that require spatially controllable composite
effects, i.e., the concurrent generation of multiple effects at designated
locations. However, integrating diverse effects into a unified framework faces
major challenges: interference from effect variations and spatial
uncontrollability during multi-VFX joint training. To tackle these challenges,
we propose Omni-Effects, a first unified framework capable of generating
prompt-guided effects and spatially controllable composite effects. The core of
our framework comprises two key innovations: (1) LoRA-based Mixture of Experts
(LoRA-MoE), which employs a group of expert LoRAs, integrating diverse effects
within a unified model while effectively mitigating cross-task interference.
(2) Spatial-Aware Prompt (SAP) incorporates spatial mask information into the
text token, enabling precise spatial control. Furthermore, we introduce an
Independent-Information Flow (IIF) module integrated within the SAP, isolating
the control signals corresponding to individual effects to prevent any unwanted
blending. To facilitate this research, we construct a comprehensive VFX dataset
Omni-VFX via a novel data collection pipeline combining image editing and
First-Last Frame-to-Video (FLF2V) synthesis, and introduce a dedicated VFX
evaluation framework for validating model performance. Extensive experiments
demonstrate that Omni-Effects achieves precise spatial control and diverse
effect generation, enabling users to specify both the category and location of
desired effects.
689ac656fab6fdd2e52ac504
Fangyuan Mao
689ac656fab6fdd2e52ac505
Aiming Hao
claimed_verified
2025-08-13T07:19:36.454Z
6583a8bdaa85c512dac3be51
/avatars/5ece6d142e85b58ea5f59b9af251ee02.svg
aiming hao
user
HamHugging
689ac656fab6fdd2e52ac506
Jintao Chen
689ac656fab6fdd2e52ac507
Dongxia Liu
689ac656fab6fdd2e52ac508
Xiaokun Feng
claimed_verified
2025-08-13T07:19:33.992Z
65e167e0d997c34a9ec59558
/avatars/aa1f38bb558c665476ac95f3cd2631eb.svg
xiaokun Feng
user
Xiaokunfeng2022
689ac656fab6fdd2e52ac509
Jiashu Zhu
689ac656fab6fdd2e52ac50a
Meiqi Wu
689ac656fab6fdd2e52ac50b
Chubin Chen
689ac656fab6fdd2e52ac50c
Jiahong Wu
689ac656fab6fdd2e52ac50d
Xiangxiang Chu
Omni-Effects: Unified and Spatially-Controllable Visual Effects
  Generation
Visual effects (VFX) are essential visual enhancements fundamental to modern
cinematic production. Although video generation models offer cost-efficient
solutions for VFX production, current methods are constrained by per-effect
LoRA training, which limits generation to single effects. This fundamental
limitation impedes applications that require spatially controllable composite
effects, i.e., the concurrent generation of multiple effects at designated
locations. However, integrating diverse effects into a unified framework faces
major challenges: interference from effect variations and spatial
uncontrollability during multi-VFX joint training. To tackle these challenges,
we propose Omni-Effects, a first unified framework capable of generating
prompt-guided effects and spatially controllable composite effects. The core of
our framework comprises two key innovations: (1) LoRA-based Mixture of Experts
(LoRA-MoE), which employs a group of expert LoRAs, integrating diverse effects
within a unified model while effectively mitigating cross-task interference.
(2) Spatial-Aware Prompt (SAP) incorporates spatial mask information into the
text token, enabling precise spatial control. Furthermore, we introduce an
Independent-Information Flow (IIF) module integrated within the SAP, isolating
the control signals corresponding to individual effects to prevent any unwanted
blending. To facilitate this research, we construct a comprehensive VFX dataset
Omni-VFX via a novel data collection pipeline combining image editing and
First-Last Frame-to-Video (FLF2V) synthesis, and introduce a dedicated VFX
evaluation framework for validating model performance. Extensive experiments
demonstrate that Omni-Effects achieves precise spatial control and diverse
effect generation, enabling users to specify both the category and location of
desired effects.
Omni-Effects is a unified framework that enables the generation of prompt-guided and spatially controllable composite visual effects using LoRA-based Mixture of Experts and Spatial-Aware Prompt with Independent-Information Flow.
LoRA Mixture of Experts LoRA-MoE Spatial-Aware Prompt SAP Independent-Information Flow IIF Omni-VFX First-Last Frame-to-Video FLF2V