mSCoRe: a Multilingual and Scalable Benchmark for Skill-based
  Commonsense Reasoning
Recent advancements in reasoning-reinforced Large Language Models (LLMs) have
shown remarkable capabilities in complex reasoning tasks. However, the
mechanism underlying their utilization of different human reasoning skills
remains poorly investigated, especially for multilingual commonsense reasoning
that involves everyday knowledge across different languages and cultures. To
address this gap, we propose a Multilingual and Scalable Benchmark for
Skill-based Commonsense Reasoning (mSCoRe).
Our benchmark incorporates three key components that are designed to
systematically evaluate LLM's reasoning capabilities, including: (1) a novel
taxonomy of reasoning skills that enables fine-grained analysis of models'
reasoning processes, (2) a robust data synthesis pipeline tailored specifically
for commonsense reasoning evaluation, and (3) a complexity scaling framework
allowing task difficulty to scale dynamically alongside future improvements in
LLM abilities. Extensive experiments on eights state-of-the-art LLMs of varying
sizes and training approaches demonstrate that mSCoRe remains
significantly challenging for current models, particularly at higher complexity
levels. Our results reveal the limitations of such reasoning-reinforced models
when confronted with nuanced multilingual general and cultural commonsense. We
further provide detailed analysis on the models' reasoning processes,
suggesting future directions for improving multilingual commonsense reasoning
capabilities.
68a6b4809e4b49496aac6aca
Nghia Trung Ngo
admin_assigned
2025-08-21T10:59:34.771Z
658a8208c4b2004663d82daf
/avatars/af6bcee06aec82602c2b931f79c008e7.svg
Nghia Trung Ngo
user
ntnghia1811
68a6b4809e4b49496aac6acb
Franck Dernoncourt
claimed_verified
2025-08-21T09:15:35.546Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
68a6b4809e4b49496aac6acc
Thien Huu Nguyen
admin_assigned
2025-08-21T10:59:41.189Z
64804fad8c6a3b8f11f73912
/avatars/61e37a91d4bba35fda9bf52aadd87745.svg
Thien Huu Nguyen
user
anoperson
mSCoRe: a Multilingual and Scalable Benchmark for Skill-based
  Commonsense Reasoning
Recent advancements in reasoning-reinforced Large Language Models (LLMs) have
shown remarkable capabilities in complex reasoning tasks. However, the
mechanism underlying their utilization of different human reasoning skills
remains poorly investigated, especially for multilingual commonsense reasoning
that involves everyday knowledge across different languages and cultures. To
address this gap, we propose a Multilingual and Scalable Benchmark for
Skill-based Commonsense Reasoning (mSCoRe).
Our benchmark incorporates three key components that are designed to
systematically evaluate LLM's reasoning capabilities, including: (1) a novel
taxonomy of reasoning skills that enables fine-grained analysis of models'
reasoning processes, (2) a robust data synthesis pipeline tailored specifically
for commonsense reasoning evaluation, and (3) a complexity scaling framework
allowing task difficulty to scale dynamically alongside future improvements in
LLM abilities. Extensive experiments on eights state-of-the-art LLMs of varying
sizes and training approaches demonstrate that mSCoRe remains
significantly challenging for current models, particularly at higher complexity
levels. Our results reveal the limitations of such reasoning-reinforced models
when confronted with nuanced multilingual general and cultural commonsense. We
further provide detailed analysis on the models' reasoning processes,
suggesting future directions for improving multilingual commonsense reasoning
capabilities.
A multilingual benchmark evaluates the reasoning skills of large language models across different languages and cultures, revealing their limitations in nuanced commonsense understanding.
Large Language Models multilingual commonsense reasoning reasoning skills data synthesis pipeline complexity scaling framework