YesBut: A High-Quality Annotated Multimodal Dataset for evaluating
  Satire Comprehension capability of Vision-Language Models
Understanding satire and humor is a challenging task for even current
Vision-Language models. In this paper, we propose the challenging tasks of
Satirical Image Detection (detecting whether an image is satirical),
Understanding (generating the reason behind the image being satirical), and
Completion (given one half of the image, selecting the other half from 2 given
options, such that the complete image is satirical) and release a high-quality
dataset YesBut, consisting of 2547 images, 1084 satirical and 1463
non-satirical, containing different artistic styles, to evaluate those tasks.
Each satirical image in the dataset depicts a normal scenario, along with a
conflicting scenario which is funny or ironic. Despite the success of current
Vision-Language Models on multimodal tasks such as Visual QA and Image
Captioning, our benchmarking experiments show that such models perform poorly
on the proposed tasks on the YesBut Dataset in Zero-Shot Settings w.r.t both
automated as well as human evaluation. Additionally, we release a dataset of
119 real, satirical photographs for further research. The dataset and code are
available at https://github.com/abhi1nandy2/yesbut_dataset.
66f18738e32a9f85b62c26d3
Abhilash Nandy
claimed_verified
2024-09-23T16:29:36.598Z
5f89da6c5d083370c711f37c
/avatars/c2f17a4a636973817fd5da2ae6dbaac3.svg
Abhilash Nandy
user
abhi1nandy2
66f18738e32a9f85b62c26d4
Yash Agarwal
admin_assigned
2024-09-23T16:31:28.934Z
66054e097c41a7efbc1fa842
/avatars/11e6c01db1f962be46a2e9122bc3066b.svg
Yash Agarwal
user
yashagarwal
66f18738e32a9f85b62c26d5
Ashish Patwa
66f18738e32a9f85b62c26d6
Millon Madhur Das
admin_assigned
2024-09-23T16:31:37.988Z
61a1284f5470d2bc8d8563e9
/avatars/04563e6f022f91c58d642c7e14c7d7d0.svg
Millon Madhur Das
user
Sleepyhead
66f18738e32a9f85b62c26d7
Aman Bansal
claimed_verified
2024-10-03T08:31:52.244Z
65c278d447ac0454b612ebdb
/avatars/63636a78b7ba289260e112603e4ee63c.svg
Aman Bansal
user
bansalaman18
66f18738e32a9f85b62c26d8
Ankit Raj
66f18738e32a9f85b62c26d9
Pawan Goyal
admin_assigned
2024-09-23T16:31:51.286Z
66d1436c55c3f7d512051c2e
/avatars/8ba01c299a244f350fc91d552192f0b5.svg
Pawan Goyal
user
pawang-iitkgp
66f18738e32a9f85b62c26da
Niloy Ganguly
YesBut: A High-Quality Annotated Multimodal Dataset for evaluating
  Satire Comprehension capability of Vision-Language Models
Understanding satire and humor is a challenging task for even current
Vision-Language models. In this paper, we propose the challenging tasks of
Satirical Image Detection (detecting whether an image is satirical),
Understanding (generating the reason behind the image being satirical), and
Completion (given one half of the image, selecting the other half from 2 given
options, such that the complete image is satirical) and release a high-quality
dataset YesBut, consisting of 2547 images, 1084 satirical and 1463
non-satirical, containing different artistic styles, to evaluate those tasks.
Each satirical image in the dataset depicts a normal scenario, along with a
conflicting scenario which is funny or ironic. Despite the success of current
Vision-Language Models on multimodal tasks such as Visual QA and Image
Captioning, our benchmarking experiments show that such models perform poorly
on the proposed tasks on the YesBut Dataset in Zero-Shot Settings w.r.t both
automated as well as human evaluation. Additionally, we release a dataset of
119 real, satirical photographs for further research. The dataset and code are
available at https://github.com/abhi1nandy2/yesbut_dataset.
Current Vision-Language models struggle with satirical image detection, understanding, and completion tasks on a newly released high-quality dataset called YesBut.
satirical image detection satirical image understanding satirical image completion Vision-Language models multimodal tasks Visual QA Image Captioning Zero-Shot Settings