DeepSeek-VL: Towards Real-World Vision-Language Understanding
We present DeepSeek-VL, an open-source Vision-Language (VL) Model designed
for real-world vision and language understanding applications. Our approach is
structured around three key dimensions:
  We strive to ensure our data is diverse, scalable, and extensively covers
real-world scenarios including web screenshots, PDFs, OCR, charts, and
knowledge-based content, aiming for a comprehensive representation of practical
contexts. Further, we create a use case taxonomy from real user scenarios and
construct an instruction tuning dataset accordingly. The fine-tuning with this
dataset substantially improves the model's user experience in practical
applications. Considering efficiency and the demands of most real-world
scenarios, DeepSeek-VL incorporates a hybrid vision encoder that efficiently
processes high-resolution images (1024 x 1024), while maintaining a relatively
low computational overhead. This design choice ensures the model's ability to
capture critical semantic and detailed information across various visual tasks.
We posit that a proficient Vision-Language Model should, foremost, possess
strong language abilities. To ensure the preservation of LLM capabilities
during pretraining, we investigate an effective VL pretraining strategy by
integrating LLM training from the beginning and carefully managing the
competitive dynamics observed between vision and language modalities.
  The DeepSeek-VL family (both 1.3B and 7B models) showcases superior user
experiences as a vision-language chatbot in real-world applications, achieving
state-of-the-art or competitive performance across a wide range of
visual-language benchmarks at the same model size while maintaining robust
performance on language-centric benchmarks. We have made both 1.3B and 7B
models publicly accessible to foster innovations based on this foundation
model.
65ee78abfc59f6e77e7e805b
Haoyu Lu
claimed_verified
2025-06-11T08:41:11.605Z
64c206a3fd4d5966b453ed85
https://cdn-avatars.huggingface.co/v1/production/uploads/64c206a3fd4d5966b453ed85/NemBrHcAJFm2ws_VQG8ia.jpeg
Haoyu Lu
user
Nealeon
65ee78abfc59f6e77e7e805c
Wen Liu
admin_assigned
2024-03-11T08:47:50.767Z
63198e8802fb322037332f2d
/avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
Wen Liu
user
doubility123
65ee78abfc59f6e77e7e805d
Bo Zhang
claimed_verified
2024-03-12T13:33:58.649Z
6237f3eb4cf571bb1ed4036b
/avatars/cbfb8ddda18e65d18a818db9f176e16e.svg
Bo Zhang
user
zhangmozhe
65ee78abfc59f6e77e7e805e
Bingxuan Wang
admin_assigned
2024-03-11T08:48:38.212Z
6523d81d56fe05f216a559f6
/avatars/07fcf56b5b8a0b64c31bdfe8fbf41cc6.svg
Bingxuan Wang
user
YellowDoge
65ee78abfc59f6e77e7e805f
Kai Dong
65ee78abfc59f6e77e7e8060
Bo Liu
claimed_verified
2024-03-18T08:36:43.367Z
635e3a76106f984574c36409
https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
Bo Liu
user
Benjamin-eecs
65ee78abfc59f6e77e7e8061
Jingxiang Sun
65ee78abfc59f6e77e7e8062
Tongzheng Ren
admin_assigned
2024-03-11T08:49:40.164Z
64f6b67db6365b4e44e5716c
/avatars/f5ef15bfa5c42e3c57f301db7efc38a1.svg
Tongzheng Ren
user
tongzheng
65ee78abfc59f6e77e7e8063
Zhuoshu Li
65ee78abfc59f6e77e7e8064
Yaofeng Sun
admin_assigned
2024-03-11T08:50:32.161Z
656b0b6cc4d6794d7f367fed
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vdJ3GU8J-9YfMJ7lI4bif.jpeg
Yaofeng Sun
user
yaofengsun
65ee78abfc59f6e77e7e8065
Chengqi Deng
65ee78abfc59f6e77e7e8066
Hanwei Xu
65ee78abfc59f6e77e7e8067
Zhenda Xie
admin_assigned
2024-03-11T08:50:54.235Z
6539f6ea26df26ecd1393c37
https://cdn-avatars.huggingface.co/v1/production/uploads/6539f6ea26df26ecd1393c37/9VJusLLAiLhUxfBAFggpF.jpeg
Zhenda Xie
user
zdaxie
65ee78abfc59f6e77e7e8068
Chong Ruan
claimed_verified
2024-08-17T05:38:38.710Z
6398203609f12714ed1935c2
https://cdn-avatars.huggingface.co/v1/production/uploads/6398203609f12714ed1935c2/uXgl0LgKnFYjq1Wz39-a6.jpeg
Chong Ruan
user
Chester111
DeepSeek-VL: Towards Real-World Vision-Language Understanding
We present DeepSeek-VL, an open-source Vision-Language (VL) Model designed
for real-world vision and language understanding applications. Our approach is
structured around three key dimensions:
  We strive to ensure our data is diverse, scalable, and extensively covers
real-world scenarios including web screenshots, PDFs, OCR, charts, and
knowledge-based content, aiming for a comprehensive representation of practical
contexts. Further, we create a use case taxonomy from real user scenarios and
construct an instruction tuning dataset accordingly. The fine-tuning with this
dataset substantially improves the model's user experience in practical
applications. Considering efficiency and the demands of most real-world
scenarios, DeepSeek-VL incorporates a hybrid vision encoder that efficiently
processes high-resolution images (1024 x 1024), while maintaining a relatively
low computational overhead. This design choice ensures the model's ability to
capture critical semantic and detailed information across various visual tasks.
We posit that a proficient Vision-Language Model should, foremost, possess
strong language abilities. To ensure the preservation of LLM capabilities
during pretraining, we investigate an effective VL pretraining strategy by
integrating LLM training from the beginning and carefully managing the
competitive dynamics observed between vision and language modalities.
  The DeepSeek-VL family (both 1.3B and 7B models) showcases superior user
experiences as a vision-language chatbot in real-world applications, achieving
state-of-the-art or competitive performance across a wide range of
visual-language benchmarks at the same model size while maintaining robust
performance on language-centric benchmarks. We have made both 1.3B and 7B
models publicly accessible to foster innovations based on this foundation
model.
DeepSeek-VL is an open-source vision-language model that achieves state-of-the-art performance in real-world applications by combining a hybrid vision encoder with effective pretraining strategies to preserve language model capabilities.
DeepSeek-VL Vision-Language Model use case taxonomy instruction tuning dataset fine-tuning hybrid vision encoder LLM training competitive dynamics vision-language benchmarks language-centric benchmarks