M^3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial
  Meeting Understanding Evaluation Dataset
Recent breakthroughs in large language models (LLMs) have led to the
development of new benchmarks for evaluating their performance in the financial
domain. However, current financial benchmarks often rely on news articles,
earnings reports, or announcements, making it challenging to capture the
real-world dynamics of financial meetings. To address this gap, we propose a
novel benchmark called M^3FinMeeting, which is a multilingual,
multi-sector, and multi-task dataset designed for financial meeting
understanding. First, M^3FinMeeting supports English, Chinese, and
Japanese, enhancing comprehension of financial discussions in diverse
linguistic contexts. Second, it encompasses various industry sectors defined by
the Global Industry Classification Standard (GICS), ensuring that the benchmark
spans a broad range of financial activities. Finally,
M^3FinMeeting includes three tasks: summarization, question-answer
(QA) pair extraction, and question answering, facilitating a more realistic and
comprehensive evaluation of understanding. Experimental results with seven
popular LLMs reveal that even the most advanced long-context models have
significant room for improvement, demonstrating the effectiveness of
M^3FinMeeting as a benchmark for assessing LLMs' financial meeting
comprehension skills.
683faa31f0564d1fb4b9ffc6
Jie Zhu
claimed_verified
2025-06-04T08:57:01.885Z
642656cbad1e3b0e6e91b752
/avatars/3bf0ee15fd528e09b2b889f5cce3cbd0.svg
Jie Zhu
user
amazingj
683faa31f0564d1fb4b9ffc7
Junhui Li
683faa31f0564d1fb4b9ffc8
Yalong Wen
admin_assigned
2025-06-05T07:38:27.545Z
64be2b4fc733e8552fea3fab
/avatars/661efe278fe5885715c8fe27129d285b.svg
yalong wen
user
yalongwen
683faa31f0564d1fb4b9ffc9
Xiandong Li
admin_assigned
2025-06-05T07:38:34.119Z
68382360c537d91527480f2f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/WGDHKjEdsWZp9nA77Zehc.png
Xiandong Li
user
Diracket
683faa31f0564d1fb4b9ffca
Lifan Guo
admin_assigned
2025-06-05T07:38:20.620Z
62f0bb7f47cf7b3344fe9ad6
/avatars/e509f31af551cfd30e319b03d4c92740.svg
guo
user
lifan
683faa31f0564d1fb4b9ffcb
Feng Chen
M^3FinMeeting: A Multilingual, Multi-Sector, and Multi-Task Financial
  Meeting Understanding Evaluation Dataset
Recent breakthroughs in large language models (LLMs) have led to the
development of new benchmarks for evaluating their performance in the financial
domain. However, current financial benchmarks often rely on news articles,
earnings reports, or announcements, making it challenging to capture the
real-world dynamics of financial meetings. To address this gap, we propose a
novel benchmark called M^3FinMeeting, which is a multilingual,
multi-sector, and multi-task dataset designed for financial meeting
understanding. First, M^3FinMeeting supports English, Chinese, and
Japanese, enhancing comprehension of financial discussions in diverse
linguistic contexts. Second, it encompasses various industry sectors defined by
the Global Industry Classification Standard (GICS), ensuring that the benchmark
spans a broad range of financial activities. Finally,
M^3FinMeeting includes three tasks: summarization, question-answer
(QA) pair extraction, and question answering, facilitating a more realistic and
comprehensive evaluation of understanding. Experimental results with seven
popular LLMs reveal that even the most advanced long-context models have
significant room for improvement, demonstrating the effectiveness of
M^3FinMeeting as a benchmark for assessing LLMs' financial meeting
comprehension skills.
A new multilingual, multi-sector, and multi-task benchmark, M³FinMeeting, evaluates large language models' performance in understanding financial meetings across different languages and industries.
large language models multilingual multi-sector multi-task benchmark financial meeting understanding Global Industry Classification Standard (GICS) summarization question-answer pair extraction question answering