MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for
  N-level Assessment
Recent advancements in large vision-language models (VLMs) have primarily
focused on English, with limited attention given to other languages. To address
this gap, we introduce MEENA (also known as PersianMMMU), the first dataset
designed to evaluate Persian VLMs across scientific, reasoning, and human-level
understanding tasks. Our dataset comprises approximately 7,500 Persian and
3,000 English questions, covering a wide range of topics such as reasoning,
mathematics, physics, diagrams, charts, and Persian art and literature. Key
features of MEENA include: (1) diverse subject coverage spanning various
educational levels, from primary to upper secondary school, (2) rich metadata,
including difficulty levels and descriptive answers, (3) original Persian data
that preserves cultural nuances, (4) a bilingual structure to assess
cross-linguistic performance, and (5) a series of diverse experiments assessing
various capabilities, including overall performance, the model's ability to
attend to images, and its tendency to generate hallucinations. We hope this
benchmark contributes to enhancing VLM capabilities beyond English.
68ad5e2586b21a0e2e358ecd
Omid Ghahroodi
claimed_verified
2025-08-26T09:49:22.462Z
6250819a5bf543dbd2607081
/avatars/dfade22a578b8349ab544acda5e8bcad.svg
Omid Ghahroodi
user
omidgh
68ad5e2586b21a0e2e358ece
Arshia Hemmat
claimed_verified
2025-08-26T19:05:52.635Z
6466bd41425875eca92aadd7
https://cdn-avatars.huggingface.co/v1/production/uploads/6466bd41425875eca92aadd7/T0OuXplQe20fNWAZUptgr.jpeg
Arshia Hemmat
user
arshiahemmat
68ad5e2586b21a0e2e358ecf
Marzia Nouri
68ad5e2586b21a0e2e358ed0
Seyed Mohammad Hadi Hosseini
claimed_verified
2025-08-29T08:03:21.243Z
6586d52b1b44d0e694f111d7
/avatars/b5e454386f795ee6454340e182970110.svg
Mohammad Hadi hosseini
user
hadihosseini0171
68ad5e2586b21a0e2e358ed1
Doratossadat Dastgheib
68ad5e2586b21a0e2e358ed2
Mohammad Vali Sanian
68ad5e2586b21a0e2e358ed3
Alireza Sahebi
claimed_verified
2025-08-26T19:05:49.119Z
61ccc2c6176cbfa298afbd58
/avatars/9f89a9190bd4cc402b0b6b0f7abefcf6.svg
Alireza Sahebi
user
AlirezaSahebi
68ad5e2586b21a0e2e358ed4
Reihaneh Zohrabi
68ad5e2586b21a0e2e358ed5
Mohammad Hossein Rohban
68ad5e2586b21a0e2e358ed6
Ehsaneddin Asgari
68ad5e2586b21a0e2e358ed7
Mahdieh Soleymani Baghshah
MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for
  N-level Assessment
Recent advancements in large vision-language models (VLMs) have primarily
focused on English, with limited attention given to other languages. To address
this gap, we introduce MEENA (also known as PersianMMMU), the first dataset
designed to evaluate Persian VLMs across scientific, reasoning, and human-level
understanding tasks. Our dataset comprises approximately 7,500 Persian and
3,000 English questions, covering a wide range of topics such as reasoning,
mathematics, physics, diagrams, charts, and Persian art and literature. Key
features of MEENA include: (1) diverse subject coverage spanning various
educational levels, from primary to upper secondary school, (2) rich metadata,
including difficulty levels and descriptive answers, (3) original Persian data
that preserves cultural nuances, (4) a bilingual structure to assess
cross-linguistic performance, and (5) a series of diverse experiments assessing
various capabilities, including overall performance, the model's ability to
attend to images, and its tendency to generate hallucinations. We hope this
benchmark contributes to enhancing VLM capabilities beyond English.
MEENA, a Persian-English dataset, evaluates vision-language models across scientific, reasoning, and human-level understanding tasks, enhancing capabilities beyond English.
vision-language models VLMs Persian MEENA PersianMMMU bilingual structure cross-linguistic performance hallucinations