MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
Metadata extraction is essential for cataloging and preserving datasets,
enabling effective research discovery and reproducibility, especially given the
current exponential growth in scientific research. While Masader (Alyafeai et
al.,2021) laid the groundwork for extracting a wide range of metadata
attributes from Arabic NLP datasets' scholarly articles, it relies heavily on
manual annotation. In this paper, we present MOLE, a framework that leverages
Large Language Models (LLMs) to automatically extract metadata attributes from
scientific papers covering datasets of languages other than Arabic. Our
schema-driven methodology processes entire documents across multiple input
formats and incorporates robust validation mechanisms for consistent output.
Additionally, we introduce a new benchmark to evaluate the research progress on
this task. Through systematic analysis of context length, few-shot learning,
and web browsing integration, we demonstrate that modern LLMs show promising
results in automating this task, highlighting the need for further future work
improvements to ensure consistent and reliable performance. We release the
code: https://github.com/IVUL-KAUST/MOLE and dataset:
https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.
68356e736bb42c7e99d2d266
Zaid Alyafeai
claimed_verified
2025-05-27T08:47:10.858Z
5f04bd384ec31d33a72116d1
https://cdn-avatars.huggingface.co/v1/production/uploads/1594145966049-noauth.jpeg
Zaid Alyafeai
user
Zaid
68356e736bb42c7e99d2d267
Maged S. Al-Shaibani
68356e736bb42c7e99d2d268
Bernard Ghanem
admin_assigned
2025-05-27T12:42:05.974Z
6808bf97ffadd78ec71cb721
/avatars/9adca3142c06b8f69889fcbe85fa374d.svg
Bernard Ghanem
user
bernardghanem
MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
Metadata extraction is essential for cataloging and preserving datasets,
enabling effective research discovery and reproducibility, especially given the
current exponential growth in scientific research. While Masader (Alyafeai et
al.,2021) laid the groundwork for extracting a wide range of metadata
attributes from Arabic NLP datasets' scholarly articles, it relies heavily on
manual annotation. In this paper, we present MOLE, a framework that leverages
Large Language Models (LLMs) to automatically extract metadata attributes from
scientific papers covering datasets of languages other than Arabic. Our
schema-driven methodology processes entire documents across multiple input
formats and incorporates robust validation mechanisms for consistent output.
Additionally, we introduce a new benchmark to evaluate the research progress on
this task. Through systematic analysis of context length, few-shot learning,
and web browsing integration, we demonstrate that modern LLMs show promising
results in automating this task, highlighting the need for further future work
improvements to ensure consistent and reliable performance. We release the
code: https://github.com/IVUL-KAUST/MOLE and dataset:
https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.
A framework leveraging Large Language Models (LLMs) is introduced for automatic metadata extraction from scientific papers, with a focus on datasets from languages other than Arabic.
Large Language Models (LLMs) schema-driven methodology benchmark few-shot learning web browsing integration