MiniGPT-v2: large language model as a unified interface for
  vision-language multi-task learning
Large language models have shown their remarkable capabilities as a general
interface for various language-related applications. Motivated by this, we
target to build a unified interface for completing many vision-language tasks
including image description, visual question answering, and visual grounding,
among others. The challenge is to use a single model for performing diverse
vision-language tasks effectively with simple multi-modal instructions. Towards
this objective, we introduce MiniGPT-v2, a model that can be treated as a
unified interface for better handling various vision-language tasks. We propose
using unique identifiers for different tasks when training the model. These
identifiers enable our model to better distinguish each task instruction
effortlessly and also improve the model learning efficiency for each task.
After the three-stage training, the experimental results show that MiniGPT-v2
achieves strong performance on many visual question-answering and visual
grounding benchmarks compared to other vision-language generalist models. Our
model and codes are available at https://minigpt-v2.github.io/
652dea01cc4609d64398ca85
Jun Chen
claimed_verified
2023-10-17T07:30:56.960Z
62f94ec077b722f186641192
/avatars/37f4c7f2b6f59e2eac2d1b1b5cc88269.svg
jun chen
user
junchen14
652dea01cc4609d64398ca86
Deyao Zhu
652dea01cc4609d64398ca87
Xiaoqian Shen
admin_assigned
2023-10-17T07:41:03.238Z
620c5b20c33ca69c4c82145b
/avatars/f3b532c47e434be30d1005fe6d0e25fb.svg
Xiaoqian Shen
user
shenxq
652dea01cc4609d64398ca88
Xiang Li
claimed_verified
2023-10-17T07:49:43.602Z
63fb35b01b4b1bd4e7fd1c72
/avatars/e3c77e8afff06af4ee6dd025c16feac7.svg
Xiang Li
user
xiang709
652dea01cc4609d64398ca89
Zechun Liu
admin_assigned
2023-10-17T07:41:21.467Z
620ae55fe190de4b8f27ba9d
/avatars/aaa2d0eadbcf96c2eb9059e3d73c2760.svg
Liu
user
Zechun
652dea01cc4609d64398ca8a
Pengchuan Zhang
admin_assigned
2023-10-17T07:41:28.897Z
640056f5261cfa61f39b4c35
/avatars/25aa5dee42caa69327d158a669f6bf3a.svg
Pengchuan Zhang
user
pzzhang
652dea01cc4609d64398ca8b
Raghuraman Krishnamoorthi
652dea01cc4609d64398ca8c
Vikas Chandra
652dea01cc4609d64398ca8d
Yunyang Xiong
652dea01cc4609d64398ca8e
Mohamed Elhoseiny
admin_assigned
2023-10-17T07:42:01.590Z
64a27d39649b0d08ca0a4ca6
/avatars/e4446a875506c10de9ae28411dc6416d.svg
Mohamed Elhoseiny
user
mhelhoseiny
MiniGPT-v2: large language model as a unified interface for
  vision-language multi-task learning
Large language models have shown their remarkable capabilities as a general
interface for various language-related applications. Motivated by this, we
target to build a unified interface for completing many vision-language tasks
including image description, visual question answering, and visual grounding,
among others. The challenge is to use a single model for performing diverse
vision-language tasks effectively with simple multi-modal instructions. Towards
this objective, we introduce MiniGPT-v2, a model that can be treated as a
unified interface for better handling various vision-language tasks. We propose
using unique identifiers for different tasks when training the model. These
identifiers enable our model to better distinguish each task instruction
effortlessly and also improve the model learning efficiency for each task.
After the three-stage training, the experimental results show that MiniGPT-v2
achieves strong performance on many visual question-answering and visual
grounding benchmarks compared to other vision-language generalist models. Our
model and codes are available at https://minigpt-v2.github.io/
MiniGPT-v2 is a unified vision-language model that uses task-specific identifiers to perform diverse tasks with high performance.
MiniGPT-v2 vision-language tasks image description visual question answering visual grounding unique identifiers multistage training visual question-answering benchmarks visual grounding benchmarks