A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics
Multi-hop Machine reading comprehension is a challenging task with aim of
answering a question based on disjoint pieces of information across the
different passages. The evaluation metrics and datasets are a vital part of
multi-hop MRC because it is not possible to train and evaluate models without
them, also, the proposed challenges by datasets often are an important
motivation for improving the existing models. Due to increasing attention to
this field, it is necessary and worth reviewing them in detail. This study aims
to present a comprehensive survey on recent advances in multi-hop MRC
evaluation metrics and datasets. In this regard, first, the multi-hop MRC
problem definition will be presented, then the evaluation metrics based on
their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have
been reviewed in detail from 2017 to 2022, and a comprehensive analysis has
been prepared at the end. Finally, open issues in this field have been
discussed.