Toward Interactive Dictation
Voice dictation is an increasingly important text input modality. Existing
systems that allow both dictation and editing-by-voice restrict their command
language to flat templates invoked by trigger words. In this work, we study the
feasibility of allowing users to interrupt their dictation with spoken editing
commands in open-ended natural language. We introduce a new task and dataset,
TERTiUS, to experiment with such systems. To support this flexibility in
real-time, a system must incrementally segment and classify spans of speech as
either dictation or command, and interpret the spans that are commands. We
experiment with using large pre-trained language models to predict the edited
text, or alternatively, to predict a small text-editing program. Experiments
show a natural trade-off between model accuracy and latency: a smaller model
achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger
model achieves 55% end-state accuracy with 7 seconds of latency.
64acb1286864362a7efb463f
Belinda Z. Li
64acb1286864362a7efb4640
Jason Eisner
64acb1286864362a7efb4641
Adam Pauls
64acb1286864362a7efb4642
Sam Thomson
claimed_verified
2023-07-13T08:32:59.130Z
64aed77821d18d09271708b7
/avatars/1c014f0c48fc85edb8e8e28a1e8bb8ae.svg
Samuel M Thomson
user
sammthomson
Toward Interactive Dictation
Voice dictation is an increasingly important text input modality. Existing
systems that allow both dictation and editing-by-voice restrict their command
language to flat templates invoked by trigger words. In this work, we study the
feasibility of allowing users to interrupt their dictation with spoken editing
commands in open-ended natural language. We introduce a new task and dataset,
TERTiUS, to experiment with such systems. To support this flexibility in
real-time, a system must incrementally segment and classify spans of speech as
either dictation or command, and interpret the spans that are commands. We
experiment with using large pre-trained language models to predict the edited
text, or alternatively, to predict a small text-editing program. Experiments
show a natural trade-off between model accuracy and latency: a smaller model
achieves 30% end-state accuracy with 1.3 seconds of latency, while a larger
model achieves 55% end-state accuracy with 7 seconds of latency.
A study explores real-time spoken interruption for dictation and editing using large pre-trained language models, showing a trade-off between accuracy and latency.
large pre-trained language models end-state accuracy latency real-time spoken interruption dictation editing commands natural language