DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language
  Models' Understanding on Indian Culture
We introduce DRISHTIKON, a first-of-its-kind multimodal and multilingual
benchmark centered exclusively on Indian culture, designed to evaluate the
cultural understanding of generative AI systems. Unlike existing benchmarks
with a generic or global scope, DRISHTIKON offers deep, fine-grained coverage
across India's diverse regions, spanning 15 languages, covering all states and
union territories, and incorporating over 64,000 aligned text-image pairs. The
dataset captures rich cultural themes including festivals, attire, cuisines,
art forms, and historical heritage amongst many more. We evaluate a wide range
of vision-language models (VLMs), including open-source small and large models,
proprietary systems, reasoning-specialized VLMs, and Indic-focused models,
across zero-shot and chain-of-thought settings. Our results expose key
limitations in current models' ability to reason over culturally grounded,
multimodal inputs, particularly for low-resource languages and less-documented
traditions. DRISHTIKON fills a vital gap in inclusive AI research, offering a
robust testbed to advance culturally aware, multimodally competent language
technologies.
68d3e8ae4402066be56e1a99
Arijit Maji
admin_assigned
2025-09-24T15:30:28.360Z
66b725125ce4c02bda63e7f0
/avatars/6f8c6b0bbfdb2d429263dea95ab32bea.svg
Arijit Maji
user
13ari
68d3e8ae4402066be56e1a9a
Raghvendra Kumar
admin_assigned
2025-09-24T15:30:35.497Z
6790ea9a766b27e112b9fc1a
/avatars/72502cc892c90ee2e9b825aab94ba9a0.svg
Raghvendra Kumar
user
procodz
68d3e8ae4402066be56e1a9b
Akash Ghosh
68d3e8ae4402066be56e1a9c
Anushka
68d3e8ae4402066be56e1a9d
Nemil Shah
68d3e8ae4402066be56e1a9e
Abhilekh Borah
68d3e8ae4402066be56e1a9f
Vanshika Shah
68d3e8ae4402066be56e1aa0
Nishant Mishra
68d3e8ae4402066be56e1aa1
Sriparna Saha
DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language
  Models' Understanding on Indian Culture
We introduce DRISHTIKON, a first-of-its-kind multimodal and multilingual
benchmark centered exclusively on Indian culture, designed to evaluate the
cultural understanding of generative AI systems. Unlike existing benchmarks
with a generic or global scope, DRISHTIKON offers deep, fine-grained coverage
across India's diverse regions, spanning 15 languages, covering all states and
union territories, and incorporating over 64,000 aligned text-image pairs. The
dataset captures rich cultural themes including festivals, attire, cuisines,
art forms, and historical heritage amongst many more. We evaluate a wide range
of vision-language models (VLMs), including open-source small and large models,
proprietary systems, reasoning-specialized VLMs, and Indic-focused models,
across zero-shot and chain-of-thought settings. Our results expose key
limitations in current models' ability to reason over culturally grounded,
multimodal inputs, particularly for low-resource languages and less-documented
traditions. DRISHTIKON fills a vital gap in inclusive AI research, offering a
robust testbed to advance culturally aware, multimodally competent language
technologies.
DRISHTIKON is a multimodal and multilingual benchmark for evaluating generative AI systems' cultural understanding across India's diverse regions and languages.
multimodal multilingual benchmark cultural understanding generative AI systems text-image pairs vision-language models zero-shot chain-of-thought culturally grounded low-resource languages inclusive AI research culturally aware multimodally competent language technologies