Training Video Foundation Models with NVIDIA NeMo
Video Foundation Models (VFMs) have recently been used to simulate the real
world to train physical AI systems and develop creative visual experiences.
However, there are significant challenges in training large-scale, high quality
VFMs that can generate high-quality videos. We present a scalable, open-source
VFM training pipeline with NVIDIA NeMo, providing accelerated video dataset
curation, multimodal data loading, and parallelized video diffusion model
training and inference. We also provide a comprehensive performance analysis
highlighting best practices for efficient VFM training and inference.
67d8d29fbe55b912664641d7
Zeeshan Patel
claimed_verified
2025-03-25T08:21:30.699Z
638ee4df6251c8bd7abd246e
/avatars/8d8c4dac7d82cba6dcf884460bdc5328.svg
Zeeshan Patel
user
zeeshanp
67d8d29fbe55b912664641d8
Ethan He
claimed_verified
2025-04-25T08:36:02.115Z
62b20f6df4a72794189248fc
/avatars/87e1125868616d4f7d6ee1e5ec4499b4.svg
Ethan He
user
ethanhe
67d8d29fbe55b912664641d9
Parth Mannan
67d8d29fbe55b912664641da
Xiaowei Ren
67d8d29fbe55b912664641db
Ryan Wolf
67d8d29fbe55b912664641dc
Niket Agarwal
67d8d29fbe55b912664641dd
Jacob Huffman
67d8d29fbe55b912664641de
Zhuoyao Wang
67d8d29fbe55b912664641df
Carl Wang
67d8d29fbe55b912664641e0
Jack Chang
67d8d29fbe55b912664641e1
Yan Bai
67d8d29fbe55b912664641e2
Tommy Huang
67d8d29fbe55b912664641e3
Linnan Wang
67d8d29fbe55b912664641e4
Sahil Jain
67d8d29fbe55b912664641e5
Shanmugam Ramasamy
67d8d29fbe55b912664641e6
Joseph Jennings
67d8d29fbe55b912664641e7
Ekaterina Sirazitdinova
67d8d29fbe55b912664641e8
Oleg Sudakov
67d8d29fbe55b912664641e9
Mingyuan Ma
67d8d29fbe55b912664641ea
Bobby Chen
67d8d29fbe55b912664641eb
Forrest Lin
67d8d29fbe55b912664641ec
Hao Wang
67d8d29fbe55b912664641ed
Vasanth Rao Naik Sabavat
67d8d29fbe55b912664641ee
Sriharsha Niverty
67d8d29fbe55b912664641ef
Rong Ou
67d8d29fbe55b912664641f0
Pallab Bhattacharya
67d8d29fbe55b912664641f1
David Page
67d8d29fbe55b912664641f2
Nima Tajbakhsh
67d8d29fbe55b912664641f3
Ashwath Aithal
Training Video Foundation Models with NVIDIA NeMo
Video Foundation Models (VFMs) have recently been used to simulate the real
world to train physical AI systems and develop creative visual experiences.
However, there are significant challenges in training large-scale, high quality
VFMs that can generate high-quality videos. We present a scalable, open-source
VFM training pipeline with NVIDIA NeMo, providing accelerated video dataset
curation, multimodal data loading, and parallelized video diffusion model
training and inference. We also provide a comprehensive performance analysis
highlighting best practices for efficient VFM training and inference.
A scalable open-source pipeline using NVIDIA NeMo for training and inference of Video Foundation Models addresses challenges in generating high-quality videos.
Video Foundation Models VFM NVIDIA NeMo video diffusion model multimodal data loading parallelized training performance analysis efficient training