LLaVA-OneVision: Easy Visual Task Transfer
We present LLaVA-OneVision, a family of open large multimodal models (LMMs)
developed by consolidating our insights into data, models, and visual
representations in the LLaVA-NeXT blog series. Our experimental results
demonstrate that LLaVA-OneVision is the first single model that can
simultaneously push the performance boundaries of open LMMs in three important
computer vision scenarios: single-image, multi-image, and video scenarios.
Importantly, the design of LLaVA-OneVision allows strong transfer learning
across different modalities/scenarios, yielding new emerging capabilities. In
particular, strong video understanding and cross-scenario capabilities are
demonstrated through task transfer from images to videos.
66b2deea8188b0d6f73f4d95
Bo Li
claimed_verified
2024-08-09T07:48:21.928Z
62d3f7d84b0933c48f3cdd9c
https://cdn-avatars.huggingface.co/v1/production/uploads/62d3f7d84b0933c48f3cdd9c/rSoSyH0Td9fD9HVlyK7bh.jpeg
Bo Li
user
luodian
66b2deea8188b0d6f73f4d96
Yuanhan Zhang
admin_assigned
2024-08-08T10:58:10.922Z
62a993d80472c0b7f94027df
https://cdn-avatars.huggingface.co/v1/production/uploads/62a993d80472c0b7f94027df/j5vp-IwLA2YBexylUHiQU.png
Zhang Yuanhan
user
ZhangYuanhan
66b2deea8188b0d6f73f4d97
Dong Guo
66b2deea8188b0d6f73f4d98
Renrui Zhang
66b2deea8188b0d6f73f4d99
Feng Li
66b2deea8188b0d6f73f4d9a
Hao Zhang
66b2deea8188b0d6f73f4d9b
Kaichen Zhang
claimed_verified
2024-08-08T07:40:31.239Z
64bb77e786e7fb5b8a317a43
https://cdn-avatars.huggingface.co/v1/production/uploads/64bb77e786e7fb5b8a317a43/J0jOrlZJ9gazdYaeSH2Bo.png
kcz
user
kcz358
66b2deea8188b0d6f73f4d9c
Yanwei Li
admin_assigned
2024-08-08T10:59:21.608Z
643ff78cdc984afcbbbc3b1a
/avatars/eec5198ce88aaf8156840bec0d190a7f.svg
Yanwei Li
user
YanweiLi
66b2deea8188b0d6f73f4d9d
Ziwei Liu
admin_assigned
2024-08-08T10:59:13.559Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
66b2deea8188b0d6f73f4d9e
Chunyuan Li
admin_assigned
2024-08-08T10:59:02.955Z
62aba526cae4462c0c6caa0f
/avatars/430560ec2c2547f819225769ab432f30.svg
Chunyuan Li
user
Chunyuan24
LLaVA-OneVision: Easy Visual Task Transfer
We present LLaVA-OneVision, a family of open large multimodal models (LMMs)
developed by consolidating our insights into data, models, and visual
representations in the LLaVA-NeXT blog series. Our experimental results
demonstrate that LLaVA-OneVision is the first single model that can
simultaneously push the performance boundaries of open LMMs in three important
computer vision scenarios: single-image, multi-image, and video scenarios.
Importantly, the design of LLaVA-OneVision allows strong transfer learning
across different modalities/scenarios, yielding new emerging capabilities. In
particular, strong video understanding and cross-scenario capabilities are
demonstrated through task transfer from images to videos.
LLaVA-OneVision is a unified multimodal model that advances performance across single-image, multi-image, and video scenarios with strong transfer learning capabilities.
multimodal models LMMs LLaVA-OneVision single-image multi-image video scenarios transfer learning video understanding cross-scenario capabilities task transfer