GRS-QA -- Graph Reasoning-Structured Question Answering Dataset
Large Language Models (LLMs) have excelled in multi-hop question-answering
(M-QA) due to their advanced reasoning abilities. However, the impact of the
inherent reasoning structures on LLM M-QA performance remains unclear, largely
due to the absence of QA datasets that provide fine-grained reasoning
structures. To address this gap, we introduce the Graph Reasoning-Structured
Question Answering Dataset (GRS-QA), which includes both semantic contexts and
reasoning structures for QA pairs. Unlike existing M-QA datasets, where
different reasoning structures are entangled together, GRS-QA explicitly
captures intricate reasoning pathways by constructing reasoning graphs, where
nodes represent textual contexts and edges denote logical flows. These
reasoning graphs of different structures enable a fine-grained evaluation of
LLM reasoning capabilities across various reasoning structures. Our empirical
analysis reveals that LLMs perform differently when handling questions with
varying reasoning structures. This finding facilitates the exploration of
textual structures as compared with semantics.
67284aa68cee5fc9fcce8804
Anish Pahilajani
admin_assigned
2024-11-04T21:35:11.622Z
6392a8f56154b82600e9e6ea
/avatars/0c2d399e42f3d4dee194d217259739fa.svg
Anish Pahilajani
user
Anish13
67284aa68cee5fc9fcce8805
Devasha Trivedi
admin_assigned
2024-11-04T21:35:17.358Z
64dea11a2aeee679693748fe
https://cdn-avatars.huggingface.co/v1/production/uploads/64dea11a2aeee679693748fe/hlew1rksek7tmMBK5P-9g.jpeg
Devasha Trivedi
user
devashat
67284aa68cee5fc9fcce8806
Jincen Shuai
admin_assigned
2024-11-04T21:35:23.196Z
65dd26e17a14b83ee0d62d8f
/avatars/c240e81a73f4025ae3645037a0847325.svg
Jincen Shuai
user
afafugai
67284aa68cee5fc9fcce8807
Khin S. Yone
67284aa68cee5fc9fcce8808
Samyak Rajesh Jain
67284aa68cee5fc9fcce8809
Namyong Park
extracted_pending
2024-11-04T04:16:42.242Z
65f9ecb65ff1f80a69598e33
/avatars/e0dd617533d9d7d6167a8dc4383ae08e.svg
Namyong Park
user
namyongp
67284aa68cee5fc9fcce880a
Ryan A. Rossi
admin_assigned
2024-11-04T21:35:48.243Z
62a3ab83e4dd6252344d27cd
/avatars/7ca8510f70a58dc207b104240e30c35c.svg
Ryan A. Rossi
user
ryanrossi
67284aa68cee5fc9fcce880b
Nesreen K. Ahmed
admin_assigned
2024-11-04T21:35:56.985Z
663e588ccca86c8371a913d9
https://cdn-avatars.huggingface.co/v1/production/uploads/663e588ccca86c8371a913d9/wgVGw5U77cqwtjovhsXyR.jpeg
Nesreen Ahmed
user
nkahmed
67284aa68cee5fc9fcce880c
Franck Dernoncourt
extracted_confirmed
2024-11-04T04:18:15.840Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
67284aa68cee5fc9fcce880d
Yu Wang
GRS-QA -- Graph Reasoning-Structured Question Answering Dataset
Large Language Models (LLMs) have excelled in multi-hop question-answering
(M-QA) due to their advanced reasoning abilities. However, the impact of the
inherent reasoning structures on LLM M-QA performance remains unclear, largely
due to the absence of QA datasets that provide fine-grained reasoning
structures. To address this gap, we introduce the Graph Reasoning-Structured
Question Answering Dataset (GRS-QA), which includes both semantic contexts and
reasoning structures for QA pairs. Unlike existing M-QA datasets, where
different reasoning structures are entangled together, GRS-QA explicitly
captures intricate reasoning pathways by constructing reasoning graphs, where
nodes represent textual contexts and edges denote logical flows. These
reasoning graphs of different structures enable a fine-grained evaluation of
LLM reasoning capabilities across various reasoning structures. Our empirical
analysis reveals that LLMs perform differently when handling questions with
varying reasoning structures. This finding facilitates the exploration of
textual structures as compared with semantics.
GRS-QA, a new dataset providing fine-grained reasoning structures, enables detailed evaluation of LLMs across various reasoning pathways in multi-hop question-answering tasks.
Large Language Models multi-hop question-answering reasoning structures Graph Reasoning-Structured Question Answering Dataset reasoning graphs textual contexts logical flows