InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding
  and Generation
This paper introduces InternVid, a large-scale video-centric multimodal
dataset that enables learning powerful and transferable video-text
representations for multimodal understanding and generation. The InternVid
dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M
video clips accompanied by detailed descriptions of total 4.1B words. Our core
contribution is to develop a scalable approach to autonomously build a
high-quality video-text dataset with large language models (LLM), thereby
showcasing its efficacy in learning video-language representation at scale.
Specifically, we utilize a multi-scale approach to generate video-related
descriptions. Furthermore, we introduce ViCLIP, a video-text representation
learning model based on ViT-L. Learned on InternVid via contrastive learning,
this model demonstrates leading zero-shot action recognition and competitive
video retrieval performance. Beyond basic video understanding tasks like
recognition and retrieval, our dataset and model have broad applications. They
are particularly beneficial for generating interleaved video-text data for
learning a video-centric dialogue system, advancing video-to-text and
text-to-video generation research. These proposed resources provide a tool for
researchers and practitioners interested in multimodal video understanding and
generation.
64b0a36ad0a5baed1da39714
Yi Wang
claimed_verified
2023-07-18T06:44:41.877Z
63119a987680dc699b2031df
/avatars/db63ab73806ee9cb8aa01be82d9effdd.svg
Yi Wang
user
shepnerd
64b0a36ad0a5baed1da39715
Yinan He
claimed_verified
2023-07-17T06:48:05.499Z
62aafa49f29ff279b51f0182
https://cdn-avatars.huggingface.co/v1/production/uploads/62aafa49f29ff279b51f0182/rQx8QFQGOY2qIhqJ8zSRj.jpeg
yinanhe
user
ynhe
64b0a36ad0a5baed1da39716
Yizhuo Li
claimed_verified
2025-06-04T09:07:09.412Z
630b094f8b327c7b8b94d24c
https://cdn-avatars.huggingface.co/v1/production/uploads/630b094f8b327c7b8b94d24c/Fd0ugLOHJZIi8oUZScOmX.jpeg
Yizhuo Li
user
liyz
64b0a36ad0a5baed1da39717
Kunchang Li
admin_assigned
2023-07-17T10:15:57.763Z
61fb81006374891646732f37
https://cdn-avatars.huggingface.co/v1/production/uploads/1643872995181-61fb81006374891646732f37.jpeg
Kunchang Li
user
Andy1621
64b0a36ad0a5baed1da39718
Jiashuo Yu
claimed_verified
2023-09-19T11:30:28.806Z
64b89a14cf14c2fabe96664c
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tEd3fBjMcEubF4plqzcUz.jpeg
Jiashuo Yu
user
awojustin
64b0a36ad0a5baed1da39719
Xin Ma
claimed_verified
2023-11-04T10:17:47.388Z
64a51faf9ec0167a99a4d114
https://cdn-avatars.huggingface.co/v1/production/uploads/64a51faf9ec0167a99a4d114/IlJS0mwkF_ZIebaQrbJIl.jpeg
Xin Ma
user
maxin-cn
64b0a36ad0a5baed1da3971a
Xinyuan Chen
admin_assigned
2023-07-17T10:19:13.580Z
643e943a70c6a27621eb1c89
/avatars/73ec521ab5ba84cc7908c52c0acef6ef.svg
Xinyuan Chen
user
AriaChen
64b0a36ad0a5baed1da3971b
Yaohui Wang
claimed_verified
2023-07-18T06:44:38.033Z
63201256c6b20f03c829c4b8
/avatars/a42092119777d65e60b12eb5ba0e45f1.svg
Yaohui Wang
user
YaohuiW
64b0a36ad0a5baed1da3971c
Ping Luo
64b0a36ad0a5baed1da3971d
Ziwei Liu
admin_assigned
2023-07-17T10:21:18.107Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
64b0a36ad0a5baed1da3971e
Yali Wang
64b0a36ad0a5baed1da3971f
Limin Wang
admin_assigned
2023-07-17T10:22:42.928Z
62c77f4352d8ae531f5511f9
/avatars/50198ccb02ccd286975a4613fbabee28.svg
Limin Wang
user
lmwang
64b0a36ad0a5baed1da39720
Yu Qiao
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding
  and Generation
This paper introduces InternVid, a large-scale video-centric multimodal
dataset that enables learning powerful and transferable video-text
representations for multimodal understanding and generation. The InternVid
dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M
video clips accompanied by detailed descriptions of total 4.1B words. Our core
contribution is to develop a scalable approach to autonomously build a
high-quality video-text dataset with large language models (LLM), thereby
showcasing its efficacy in learning video-language representation at scale.
Specifically, we utilize a multi-scale approach to generate video-related
descriptions. Furthermore, we introduce ViCLIP, a video-text representation
learning model based on ViT-L. Learned on InternVid via contrastive learning,
this model demonstrates leading zero-shot action recognition and competitive
video retrieval performance. Beyond basic video understanding tasks like
recognition and retrieval, our dataset and model have broad applications. They
are particularly beneficial for generating interleaved video-text data for
learning a video-centric dialogue system, advancing video-to-text and
text-to-video generation research. These proposed resources provide a tool for
researchers and practitioners interested in multimodal video understanding and
generation.
InternVid, a large-scale video-text multimodal dataset, and ViCLIP, a video-text representation model based on ViT-L, enable advanced video and text understanding and generation tasks.
InternVid ViCLIP ViT-L contrastive learning zero-shot action recognition video retrieval video-centric dialogue system video-to-text generation text-to-video generation multimodal understanding multimodal generation