Bridging Text and Video Generation: A Survey
Text-to-video (T2V) generation technology holds potential to transform
multiple domains such as education, marketing, entertainment, and assistive
technologies for individuals with visual or reading comprehension challenges,
by creating coherent visual content from natural language prompts. From its
inception, the field has advanced from adversarial models to diffusion-based
models, yielding higher-fidelity, temporally consistent outputs. Yet challenges
persist, such as alignment, long-range coherence, and computational efficiency.
Addressing this evolving landscape, we present a comprehensive survey of
text-to-video generative models, tracing their development from early GANs and
VAEs to hybrid Diffusion-Transformer (DiT) architectures, detailing how these
models work, what limitations they addressed in their predecessors, and why
shifts toward new architectural paradigms were necessary to overcome challenges
in quality, coherence, and control. We provide a systematic account of the
datasets, which the surveyed text-to-video models were trained and evaluated
on, and, to support reproducibility and assess the accessibility of training
such models, we detail their training configurations, including their hardware
specifications, GPU counts, batch sizes, learning rates, optimizers, epochs,
and other key hyperparameters. Further, we outline the evaluation metrics
commonly used for evaluating such models and present their performance across
standard benchmarks, while also discussing the limitations of these metrics and
the emerging shift toward more holistic, perception-aligned evaluation
strategies. Finally, drawing from our analysis, we outline the current open
challenges and propose a few promising future directions, laying out a
perspective for future researchers to explore and build upon in advancing T2V
research and applications.
68e548b5975ac4c405ef1f19
Nilay Kumar
claimed_verified
2025-10-08T08:01:20.349Z
63cbb330f488db9bb3be6fe6
https://cdn-avatars.huggingface.co/v1/production/uploads/63cbb330f488db9bb3be6fe6/66bg2FdIg4YSsqWzP8WAd.png
Nilay K. Bhatnagar
user
nnilayy
68e548b5975ac4c405ef1f1a
Priyansh Bhandari
68e548b5975ac4c405ef1f1b
G. Maragatham
Bridging Text and Video Generation: A Survey
Text-to-video (T2V) generation technology holds potential to transform
multiple domains such as education, marketing, entertainment, and assistive
technologies for individuals with visual or reading comprehension challenges,
by creating coherent visual content from natural language prompts. From its
inception, the field has advanced from adversarial models to diffusion-based
models, yielding higher-fidelity, temporally consistent outputs. Yet challenges
persist, such as alignment, long-range coherence, and computational efficiency.
Addressing this evolving landscape, we present a comprehensive survey of
text-to-video generative models, tracing their development from early GANs and
VAEs to hybrid Diffusion-Transformer (DiT) architectures, detailing how these
models work, what limitations they addressed in their predecessors, and why
shifts toward new architectural paradigms were necessary to overcome challenges
in quality, coherence, and control. We provide a systematic account of the
datasets, which the surveyed text-to-video models were trained and evaluated
on, and, to support reproducibility and assess the accessibility of training
such models, we detail their training configurations, including their hardware
specifications, GPU counts, batch sizes, learning rates, optimizers, epochs,
and other key hyperparameters. Further, we outline the evaluation metrics
commonly used for evaluating such models and present their performance across
standard benchmarks, while also discussing the limitations of these metrics and
the emerging shift toward more holistic, perception-aligned evaluation
strategies. Finally, drawing from our analysis, we outline the current open
challenges and propose a few promising future directions, laying out a
perspective for future researchers to explore and build upon in advancing T2V
research and applications.
A survey of text-to-video generative models from GANs and VAEs to hybrid Diffusion-Transformer architectures, detailing their development, limitations, and future directions.
adversarial models diffusion-based models GANs VAEs Diffusion-Transformer (DiT) architectures datasets training configurations evaluation metrics perception-aligned evaluation strategies