The All-Seeing Project: Towards Panoptic Visual Recognition and
  Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for
recognizing and understanding everything in the open world. Using a scalable
data engine that incorporates human feedback and efficient models in the loop,
we create a new dataset (AS-1B) with over 1 billion regions annotated with
semantic tags, question-answering pairs, and detailed captions. It covers a
wide range of 3.5 million common and rare concepts in the real world, and has
132.2 billion tokens that describe the concepts and their attributes.
Leveraging this new dataset, we develop the All-Seeing model (ASM), a unified
framework for panoptic visual recognition and understanding. The model is
trained with open-ended language prompts and locations, which allows it to
generalize to various vision and language tasks with remarkable zero-shot
performance, including region-text retrieval, region recognition, captioning,
and question-answering. We hope that this project can serve as a foundation for
vision-language artificial general intelligence research. Models and the
dataset shall be released at https://github.com/OpenGVLab/All-Seeing, and demo
can be seen at https://huggingface.co/spaces/OpenGVLab/all-seeing.
64cc847d1af278541d922dcd
Weiyun Wang
admin_assigned
2023-08-04T12:37:39.193Z
619507e7b74b6c591f794340
https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
Weiyun Wang
user
Weiyun1025
64cc847d1af278541d922dce
Min Shi
64cc847d1af278541d922dcf
Qingyun Li
admin_assigned
2023-08-04T12:39:01.086Z
637f100c1dbae09191088e4c
https://cdn-avatars.huggingface.co/v1/production/uploads/1669271529782-noauth.jpeg
Qingyun Li
user
Qingyun
64cc847d1af278541d922dd0
Wenhai Wang
admin_assigned
2023-08-04T12:39:42.218Z
644f58e6f8b353c94914c7e0
/avatars/16c7e80aafbea9a52cfc309068eedc9a.svg
wangwenhai
user
yicixi
64cc847d1af278541d922dd1
Zhenhang Huang
64cc847d1af278541d922dd2
Linjie Xing
64cc847d1af278541d922dd3
Zhe Chen
claimed_verified
2024-12-09T14:51:20.313Z
64119264f0f81eb569e0d569
/avatars/00dccedea51ac7d2f398ca9d65a8e78b.svg
Zhe Chen
user
czczup
64cc847d1af278541d922dd4
Hao Li
claimed_verified
2024-12-16T09:44:27.755Z
64b9033777ae61bcc80aa4f3
/avatars/408c335395c79f3df69fd9bf70abc312.svg
Hao Li
user
cpsxhao
64cc847d1af278541d922dd5
Xizhou Zhu
64cc847d1af278541d922dd6
Zhiguo Cao
64cc847d1af278541d922dd7
Yushi Chen
64cc847d1af278541d922dd8
Tong Lu
64cc847d1af278541d922dd9
Jifeng Dai
admin_assigned
2023-08-04T12:41:59.461Z
64686f7172d9180d4ac8b4e4
/avatars/db67dd6c4b2b41054ddcce5a18ade6f8.svg
Jifeng Dai
user
daijifeng
64cc847d1af278541d922dda
Yu Qiao
The All-Seeing Project: Towards Panoptic Visual Recognition and
  Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for
recognizing and understanding everything in the open world. Using a scalable
data engine that incorporates human feedback and efficient models in the loop,
we create a new dataset (AS-1B) with over 1 billion regions annotated with
semantic tags, question-answering pairs, and detailed captions. It covers a
wide range of 3.5 million common and rare concepts in the real world, and has
132.2 billion tokens that describe the concepts and their attributes.
Leveraging this new dataset, we develop the All-Seeing model (ASM), a unified
framework for panoptic visual recognition and understanding. The model is
trained with open-ended language prompts and locations, which allows it to
generalize to various vision and language tasks with remarkable zero-shot
performance, including region-text retrieval, region recognition, captioning,
and question-answering. We hope that this project can serve as a foundation for
vision-language artificial general intelligence research. Models and the
dataset shall be released at https://github.com/OpenGVLab/All-Seeing, and demo
can be seen at https://huggingface.co/spaces/OpenGVLab/all-seeing.
The All-Seeing project introduces a large dataset (AS-1B) and a unified model (ASM) for recognizing and understanding a wide range of concepts in the open world, achieving zero-shot performance across various vision and language tasks.
panoptic visual recognition zero-shot performance region-text retrieval region recognition captioning question-answering