The MERIT Dataset: Modelling and Efficiently Rendering Interpretable
  Transcripts
This paper introduces the MERIT Dataset, a multimodal (text + image + layout)
fully labeled dataset within the context of school reports. Comprising over 400
labels and 33k samples, the MERIT Dataset is a valuable resource for training
models in demanding Visually-rich Document Understanding (VrDU) tasks. By its
nature (student grade reports), the MERIT Dataset can potentially include
biases in a controlled way, making it a valuable tool to benchmark biases
induced in Language Models (LLMs). The paper outlines the dataset's generation
pipeline and highlights its main features in the textual, visual, layout, and
bias domains. To demonstrate the dataset's utility, we present a benchmark with
token classification models, showing that the dataset poses a significant
challenge even for SOTA models and that these would greatly benefit from
including samples from the MERIT Dataset in their pretraining phase.
66d817fcde54fea8a03973f0
I. de Rodrigo
extracted_pending
2024-09-04T08:19:10.803Z
63ee535a190ddd6214f30dc2
https://cdn-avatars.huggingface.co/v1/production/uploads/1676563169736-noauth.jpeg
de Rodrigo
user
de-Rodrigo
66d817fcde54fea8a03973f1
A. Sanchez-Cuadrado
66d817fcde54fea8a03973f2
J. Boal
66d817fcde54fea8a03973f3
A. J. Lopez-Lopez
extracted_pending
2024-09-04T08:19:10.803Z
645c966858f9ee315142a2e5
/avatars/51928db15dddc07a51cc35a402f511c7.svg
Alvaro Lopez
user
alvlopez
The MERIT Dataset: Modelling and Efficiently Rendering Interpretable
  Transcripts
This paper introduces the MERIT Dataset, a multimodal (text + image + layout)
fully labeled dataset within the context of school reports. Comprising over 400
labels and 33k samples, the MERIT Dataset is a valuable resource for training
models in demanding Visually-rich Document Understanding (VrDU) tasks. By its
nature (student grade reports), the MERIT Dataset can potentially include
biases in a controlled way, making it a valuable tool to benchmark biases
induced in Language Models (LLMs). The paper outlines the dataset's generation
pipeline and highlights its main features in the textual, visual, layout, and
bias domains. To demonstrate the dataset's utility, we present a benchmark with
token classification models, showing that the dataset poses a significant
challenge even for SOTA models and that these would greatly benefit from
including samples from the MERIT Dataset in their pretraining phase.
MerIT Dataset, containing text, image, and layout data from school reports, benchmarks biases in language models and poses challenges for state-of-the-art token classification models.
Visually-rich Document Understanding VrDU Language Models LLMs biases token classification models pretraining phase