OpenVision 2: A Family of Generative Pretrained Visual Encoders for
  Multimodal Learning
This paper provides a simplification on OpenVision's architecture and loss
design for enhancing its training efficiency. Following the prior
vision-language pretraining works CapPa and AIMv2, as well as modern multimodal
designs like LLaVA, our changes are straightforward: we remove the text encoder
(and therefore the contrastive loss), retaining only the captioning loss as a
purely generative training signal. We name this new version OpenVision 2. The
initial results are promising: despite this simplification, OpenVision 2
competitively matches the original model's performance on a broad set of
multimodal benchmarks while substantially cutting both training time and memory
consumption. For example, with ViT-L/14, it reduces training time by about 1.5x
(from 83h to 57h), and memory usage by about 1.8x (from 24.5GB to 13.8GB,
equivalently allowing the maximum batch size to grow from 2k to 8k). This
superior training efficiency also allows us to scale far beyond the largest
vision encoder used in OpenVision, reaching more than 1 billion parameters. We
hold a strong belief that this lightweight, generative-only paradigm is
compelling for future vision encoder development in multimodal foundation
models.
68b7beee295f15ff609113c2
Yanqing Liu
claimed_verified
2025-09-04T08:44:19.655Z
654a771a2d2fcd6bf27ff0de
/avatars/6312b23c642009f13c7993ce4658d06d.svg
Liuyanqing
user
Yanqing0327
68b7beee295f15ff609113c3
Xianhang Li
68b7beee295f15ff609113c4
Letian Zhang
68b7beee295f15ff609113c5
Zirui Wang
68b7beee295f15ff609113c6
Zeyu Zheng
68b7beee295f15ff609113c7
Yuyin Zhou
68b7beee295f15ff609113c8
Cihang Xie
claimed_verified
2025-09-03T08:26:35.974Z
645eb61da3c5cd8a16efffff
/avatars/9112bfeed598dfabf9e077e69e09ecc9.svg
Cihang Xie
user
cihangxie
OpenVision 2: A Family of Generative Pretrained Visual Encoders for
  Multimodal Learning
This paper provides a simplification on OpenVision's architecture and loss
design for enhancing its training efficiency. Following the prior
vision-language pretraining works CapPa and AIMv2, as well as modern multimodal
designs like LLaVA, our changes are straightforward: we remove the text encoder
(and therefore the contrastive loss), retaining only the captioning loss as a
purely generative training signal. We name this new version OpenVision 2. The
initial results are promising: despite this simplification, OpenVision 2
competitively matches the original model's performance on a broad set of
multimodal benchmarks while substantially cutting both training time and memory
consumption. For example, with ViT-L/14, it reduces training time by about 1.5x
(from 83h to 57h), and memory usage by about 1.8x (from 24.5GB to 13.8GB,
equivalently allowing the maximum batch size to grow from 2k to 8k). This
superior training efficiency also allows us to scale far beyond the largest
vision encoder used in OpenVision, reaching more than 1 billion parameters. We
hold a strong belief that this lightweight, generative-only paradigm is
compelling for future vision encoder development in multimodal foundation
models.
OpenVision 2 simplifies the original architecture by removing the text encoder and contrastive loss, achieving competitive performance with reduced training time and memory usage, and enabling scaling to over 1 billion parameters.
OpenVision text encoder contrastive loss captioning loss multimodal benchmarks training time memory consumption ViT-L/14 batch size vision encoder multimodal foundation models