POINTS1.5: Building a Vision-Language Model towards Real World
  Applications
Vision-language models have made significant strides recently, demonstrating
superior performance across a range of tasks, e.g. optical character
recognition and complex diagram analysis. Building on this trend, we introduce
a new vision-language model, POINTS1.5, designed to excel in various real-world
applications. POINTS1.5 is an enhancement of POINTS1.0 and incorporates several
key innovations: i) We replace the original CLIP vision encoder, which had a
fixed image resolution, with a NaViT-style vision encoder that supports native
dynamic high resolution. This allows POINTS1.5 to process images of any
resolution without needing to split them into tiles. ii) We add bilingual
support to POINTS1.5, significantly enhancing its capability in Chinese. Due to
the scarcity of open-source Chinese datasets for vision-language models, we
collect numerous images from the Internet and annotate them using a combination
of manual and automatic methods. iii) We propose a set of rigorous filtering
methods for visual instruction tuning datasets. We comprehensively evaluate all
these filtering methods, and choose the most effective ones to obtain the final
visual instruction tuning set. Thanks to these innovations, POINTS1.5
significantly outperforms POINTS1.0 and demonstrates strong performance across
a range of real-world applications. Notably, POINTS1.5-7B is trained on fewer
than 4 billion tokens and ranks first on the OpenCompass leaderboard among
models with fewer than 10 billion parameters
675a62917f9a1b78132efc33
Yuan Liu
admin_assigned
2024-12-12T08:28:51.562Z
64d47a7a508a6313e33faedd
https://cdn-avatars.huggingface.co/v1/production/uploads/64d47a7a508a6313e33faedd/8DrbXdxA-sKZb4jgNhvY1.jpeg
Yuan Liu
user
YuanLiuuuuuu
675a62917f9a1b78132efc34
Le Tian
675a62917f9a1b78132efc35
Xiao Zhou
675a62917f9a1b78132efc36
Xinyu Gao
675a62917f9a1b78132efc37
Kavio Yu
admin_assigned
2024-12-12T08:29:26.282Z
663f288e9cb0add00b96bf45
/avatars/cd90b9490c439703cb3b52ccbc491199.svg
kavioyu
user
kavio
675a62917f9a1b78132efc38
Yang Yu
admin_assigned
2024-12-12T08:29:33.015Z
646aefc235c7a57f936cfa31
/avatars/fc55fadf7260f8d9e164ddafb5e2b8dc.svg
Yang Yu
user
yangyu1
675a62917f9a1b78132efc39
Jie Zhou
POINTS1.5: Building a Vision-Language Model towards Real World
  Applications
Vision-language models have made significant strides recently, demonstrating
superior performance across a range of tasks, e.g. optical character
recognition and complex diagram analysis. Building on this trend, we introduce
a new vision-language model, POINTS1.5, designed to excel in various real-world
applications. POINTS1.5 is an enhancement of POINTS1.0 and incorporates several
key innovations: i) We replace the original CLIP vision encoder, which had a
fixed image resolution, with a NaViT-style vision encoder that supports native
dynamic high resolution. This allows POINTS1.5 to process images of any
resolution without needing to split them into tiles. ii) We add bilingual
support to POINTS1.5, significantly enhancing its capability in Chinese. Due to
the scarcity of open-source Chinese datasets for vision-language models, we
collect numerous images from the Internet and annotate them using a combination
of manual and automatic methods. iii) We propose a set of rigorous filtering
methods for visual instruction tuning datasets. We comprehensively evaluate all
these filtering methods, and choose the most effective ones to obtain the final
visual instruction tuning set. Thanks to these innovations, POINTS1.5
significantly outperforms POINTS1.0 and demonstrates strong performance across
a range of real-world applications. Notably, POINTS1.5-7B is trained on fewer
than 4 billion tokens and ranks first on the OpenCompass leaderboard among
models with fewer than 10 billion parameters
POINTS1.5, an enhanced vision-language model, incorporates a dynamic high-resolution vision encoder, bilingual support, and rigorous dataset filtering methods to achieve superior performance.
vision-language models NaViT-style vision encoder dynamic high resolution bilingual support visual instruction tuning datasets OpenCompass leaderboard