EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video
  Guidance
Recent approaches on 3D camera control in video diffusion models (VDMs) often
create anchor videos to guide diffusion models as a structured prior by
rendering from estimated point clouds following annotated camera trajectories.
However, errors inherent in point cloud estimation often lead to inaccurate
anchor videos. Moreover, the requirement for extensive camera trajectory
annotations further increases resource demands. To address these limitations,
we introduce EPiC, an efficient and precise camera control learning framework
that automatically constructs high-quality anchor videos without expensive
camera trajectory annotations. Concretely, we create highly precise anchor
videos for training by masking source videos based on first-frame visibility.
This approach ensures high alignment, eliminates the need for camera trajectory
annotations, and thus can be readily applied to any in-the-wild video to
generate image-to-video (I2V) training pairs. Furthermore, we introduce
Anchor-ControlNet, a lightweight conditioning module that integrates anchor
video guidance in visible regions to pretrained VDMs, with less than 1% of
backbone model parameters. By combining the proposed anchor video data and
ControlNet module, EPiC achieves efficient training with substantially fewer
parameters, training steps, and less data, without requiring modifications to
the diffusion model backbone typically needed to mitigate rendering
misalignments. Although being trained on masking-based anchor videos, our
method generalizes robustly to anchor videos made with point clouds during
inference, enabling precise 3D-informed camera control. EPiC achieves SOTA
performance on RealEstate10K and MiraData for I2V camera control task,
demonstrating precise and robust camera control ability both quantitatively and
qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to
video-to-video scenarios.
6837d80bf42b2aacfc26c460
Zun Wang
6837d80bf42b2aacfc26c461
Jaemin Cho
claimed_verified
2025-07-11T16:32:06.861Z
5ffe32d8942cf3533d364449
https://cdn-avatars.huggingface.co/v1/production/uploads/1654821969191-5ffe32d8942cf3533d364449.jpeg
Jaemin Cho
user
j-min
6837d80bf42b2aacfc26c462
Jialu Li
6837d80bf42b2aacfc26c463
Han Lin
6837d80bf42b2aacfc26c464
Jaehong Yoon
claimed_verified
2025-05-29T07:42:07.590Z
652066649004117947e46ed6
/avatars/972c97df6f26d2c3d6ce71ec579984bb.svg
Jaehong Yoon
user
jaehong31
6837d80bf42b2aacfc26c465
Yue Zhang
6837d80bf42b2aacfc26c466
Mohit Bansal
EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video
  Guidance
Recent approaches on 3D camera control in video diffusion models (VDMs) often
create anchor videos to guide diffusion models as a structured prior by
rendering from estimated point clouds following annotated camera trajectories.
However, errors inherent in point cloud estimation often lead to inaccurate
anchor videos. Moreover, the requirement for extensive camera trajectory
annotations further increases resource demands. To address these limitations,
we introduce EPiC, an efficient and precise camera control learning framework
that automatically constructs high-quality anchor videos without expensive
camera trajectory annotations. Concretely, we create highly precise anchor
videos for training by masking source videos based on first-frame visibility.
This approach ensures high alignment, eliminates the need for camera trajectory
annotations, and thus can be readily applied to any in-the-wild video to
generate image-to-video (I2V) training pairs. Furthermore, we introduce
Anchor-ControlNet, a lightweight conditioning module that integrates anchor
video guidance in visible regions to pretrained VDMs, with less than 1% of
backbone model parameters. By combining the proposed anchor video data and
ControlNet module, EPiC achieves efficient training with substantially fewer
parameters, training steps, and less data, without requiring modifications to
the diffusion model backbone typically needed to mitigate rendering
misalignments. Although being trained on masking-based anchor videos, our
method generalizes robustly to anchor videos made with point clouds during
inference, enabling precise 3D-informed camera control. EPiC achieves SOTA
performance on RealEstate10K and MiraData for I2V camera control task,
demonstrating precise and robust camera control ability both quantitatively and
qualitatively. Notably, EPiC also exhibits strong zero-shot generalization to
video-to-video scenarios.
EPiC is a framework for efficient 3D camera control in video diffusion models that constructs high-quality anchor videos through first-frame visibility masking, integrates them using a lightweight ControlNet module, and achieves state-of-the-art performance on I2V tasks with minimal resources.
anchor videos point cloud estimation camera trajectories diffusion models first-frame visibility EPiC ControlNet I2V training pairs rendering misalignments RealEstate10K MiraData