RoMath: A Mathematical Reasoning Benchmark in Romanian
Mathematics has long been conveyed through natural language, primarily for
human understanding. With the rise of mechanized mathematics and proof
assistants, there is a growing need to understand informal mathematical text,
yet most existing benchmarks focus solely on English, overlooking other
languages. This paper introduces RoMath, a Romanian mathematical reasoning
benchmark suite comprising three datasets: RoMath-Baccalaureate,
RoMath-Competitions and RoMath-Synthetic, which cover a range of mathematical
domains and difficulty levels, aiming to improve non-English language models
and promote multilingual AI development. By focusing on Romanian, a
low-resource language with unique linguistic features, RoMath addresses the
limitations of Anglo-centric models and emphasizes the need for dedicated
resources beyond simple automatic translation. We benchmark several open-weight
language models, highlighting the importance of creating resources for
underrepresented languages. We make the code and dataset available.
66ead57361228b02f8144cdf
Adrian Cosma
66ead57361228b02f8144ce0
Ana-Maria Bucur
66ead57361228b02f8144ce1
Emilian Radoi
RoMath: A Mathematical Reasoning Benchmark in Romanian
Mathematics has long been conveyed through natural language, primarily for
human understanding. With the rise of mechanized mathematics and proof
assistants, there is a growing need to understand informal mathematical text,
yet most existing benchmarks focus solely on English, overlooking other
languages. This paper introduces RoMath, a Romanian mathematical reasoning
benchmark suite comprising three datasets: RoMath-Baccalaureate,
RoMath-Competitions and RoMath-Synthetic, which cover a range of mathematical
domains and difficulty levels, aiming to improve non-English language models
and promote multilingual AI development. By focusing on Romanian, a
low-resource language with unique linguistic features, RoMath addresses the
limitations of Anglo-centric models and emphasizes the need for dedicated
resources beyond simple automatic translation. We benchmark several open-weight
language models, highlighting the importance of creating resources for
underrepresented languages. We make the code and dataset available.
benchmark suite datasets mathematical reasoning benchmark benchmark low-resource language linguistic features open-weight language models multilingual AI automated mathematics proof assistants informal mathematical text code dataset synthetic baccalaureate competitions mathematical domains difficulty levels automatic translation