HeySQuAD: A Spoken Question Answering Dataset
Human-spoken questions are critical to evaluating the performance of spoken
question answering (SQA) systems that serve several real-world use cases
including digital assistants. We present a new large-scale community-shared SQA
dataset, HeySQuAD that consists of 76k human-spoken questions and 97k
machine-generated questions and corresponding textual answers derived from the
SQuAD QA dataset. The goal of HeySQuAD is to measure the ability of machines to
understand noisy spoken questions and answer the questions accurately. To this
end, we run extensive benchmarks on the human-spoken and machine-generated
questions to quantify the differences in noise from both sources and its
subsequent impact on the model and answering accuracy. Importantly, for the
task of SQA, where we want to answer human-spoken questions, we observe that
training using the transcribed human-spoken and original SQuAD questions leads
to significant improvements (12.51%) over training using only the original
SQuAD textual questions.