Using Large Language Models to Accelerate Communication for Users with
  Severe Motor Impairments
Finding ways to accelerate text input for individuals with profound motor
impairments has been a long-standing area of research. Closing the speed gap
for augmentative and alternative communication (AAC) devices such as
eye-tracking keyboards is important for improving the quality of life for such
individuals. Recent advances in neural networks of natural language pose new
opportunities for re-thinking strategies and user interfaces for enhanced
text-entry for AAC users. In this paper, we present SpeakFaster, consisting of
large language models (LLMs) and a co-designed user interface for text entry in
a highly-abbreviated form, allowing saving 57% more motor actions than
traditional predictive keyboards in offline simulation. A pilot study with 19
non-AAC participants typing on a mobile device by hand demonstrated gains in
motor savings in line with the offline simulation, while introducing relatively
small effects on overall typing speed. Lab and field testing on two eye-gaze
typing users with amyotrophic lateral sclerosis (ALS) demonstrated text-entry
rates 29-60% faster than traditional baselines, due to significant saving of
expensive keystrokes achieved through phrase and word predictions from
context-aware LLMs. These findings provide a strong foundation for further
exploration of substantially-accelerated text communication for motor-impaired
users and demonstrate a direction for applying LLMs to text-based user
interfaces.
656e98257112260ee14239ba
Shanqing Cai
admin_assigned
2023-12-05T11:16:36.621Z
63a5dfba28af3c9aa20703d7
/avatars/a47f248d01775a3ebf5819dd9367a761.svg
Shanqing Cai
user
sqcai
656e98257112260ee14239bb
Subhashini Venugopalan
extracted_confirmed
2025-03-20T13:16:59.385Z
646125a5933afb0106a9dabb
/avatars/ccef359e4442f560a87fd66228e54c8b.svg
Subhashini
user
vsubhashini
656e98257112260ee14239bc
Katie Seaver
656e98257112260ee14239bd
Xiang Xiao
656e98257112260ee14239be
Katrin Tomanek
656e98257112260ee14239bf
Sri Jalasutram
656e98257112260ee14239c0
Meredith Ringel Morris
656e98257112260ee14239c1
Shaun Kane
claimed_verified
2024-02-16T08:36:29.541Z
65ceebf9522b760d74fa7a39
https://cdn-avatars.huggingface.co/v1/production/uploads/65ceebf9522b760d74fa7a39/jv-oJlarsG86ndQo1mwDZ.png
Shaun Kane
user
shaunkane
656e98257112260ee14239c2
Ajit Narayanan
656e98257112260ee14239c3
Robert L. MacDonald
admin_assigned
2023-12-05T11:20:50.211Z
63148ab7b031f7b1c7bc2c98
/avatars/629a9a4f2725636f79c3ee3b535f6c75.svg
Robert MacDonald
user
icemunk
656e98257112260ee14239c4
Emily Kornman
656e98257112260ee14239c5
Daniel Vance
656e98257112260ee14239c6
Blair Casey
656e98257112260ee14239c7
Steve M. Gleason
656e98257112260ee14239c8
Philip Q. Nelson
admin_assigned
2023-12-05T11:20:23.962Z
6313f3bee29fb2e86d6293ae
/avatars/3d143796db99d5ae716b0e8fd35c721b.svg
Philip Nelson
user
blu3r4d0n
656e98257112260ee14239c9
Michael P. Brenner
admin_assigned
2023-12-05T11:20:12.608Z
6527116d229b584b3ef0ba0e
/avatars/2fd8c65603e81ca6084aae75520ca878.svg
Michael Brenner
user
philhankenson
Using Large Language Models to Accelerate Communication for Users with
  Severe Motor Impairments
Finding ways to accelerate text input for individuals with profound motor
impairments has been a long-standing area of research. Closing the speed gap
for augmentative and alternative communication (AAC) devices such as
eye-tracking keyboards is important for improving the quality of life for such
individuals. Recent advances in neural networks of natural language pose new
opportunities for re-thinking strategies and user interfaces for enhanced
text-entry for AAC users. In this paper, we present SpeakFaster, consisting of
large language models (LLMs) and a co-designed user interface for text entry in
a highly-abbreviated form, allowing saving 57% more motor actions than
traditional predictive keyboards in offline simulation. A pilot study with 19
non-AAC participants typing on a mobile device by hand demonstrated gains in
motor savings in line with the offline simulation, while introducing relatively
small effects on overall typing speed. Lab and field testing on two eye-gaze
typing users with amyotrophic lateral sclerosis (ALS) demonstrated text-entry
rates 29-60% faster than traditional baselines, due to significant saving of
expensive keystrokes achieved through phrase and word predictions from
context-aware LLMs. These findings provide a strong foundation for further
exploration of substantially-accelerated text communication for motor-impaired
users and demonstrate a direction for applying LLMs to text-based user
interfaces.
SpeakFaster uses large language models and a custom interface to reduce motor actions and enhance text entry speed for AAC users, particularly those with ALS.
large language models LLMs eye-tracking keyboards augmentative and alternative communication (AAC) motor actions predictive keyboards phrase prediction word prediction context-aware LLMs text-entry rates amyotrophic lateral sclerosis (ALS)