Decoding Open-Ended Information Seeking Goals from Eye Movements in
  Reading
When reading, we often have specific information that interests us in a text.
For example, you might be reading this paper because you are curious about LLMs
for eye movements in reading, the experimental design, or perhaps you only care
about the question ``but does it work?''. More broadly, in daily life, people
approach texts with any number of text-specific goals that guide their reading
behavior. In this work, we ask, for the first time, whether open-ended reading
goals can be automatically decoded from eye movements in reading. To address
this question, we introduce goal classification and goal reconstruction tasks
and evaluation frameworks, and use large-scale eye tracking for reading data in
English with hundreds of text-specific information seeking tasks. We develop
and compare several discriminative and generative multimodal LLMs that combine
eye movements and text for goal classification and goal reconstruction. Our
experiments show considerable success on both tasks, suggesting that LLMs can
extract valuable information about the readers' text-specific goals from eye
movements.
681b03a33bf166767107c74b
Cfir Avraham Hadar
681b03a33bf166767107c74c
Omer Shubi
681b03a33bf166767107c74d
Yoav Meiri
claimed_verified
2025-05-08T08:59:32.849Z
6304be28bad6ce7fc026f55f
/avatars/0597579c2f0e48f42b636997ec174004.svg
Yoav Meiri
user
YoavMeiri
681b03a33bf166767107c74e
Yevgeni Berzak
Decoding Open-Ended Information Seeking Goals from Eye Movements in
  Reading
When reading, we often have specific information that interests us in a text.
For example, you might be reading this paper because you are curious about LLMs
for eye movements in reading, the experimental design, or perhaps you only care
about the question ``but does it work?''. More broadly, in daily life, people
approach texts with any number of text-specific goals that guide their reading
behavior. In this work, we ask, for the first time, whether open-ended reading
goals can be automatically decoded from eye movements in reading. To address
this question, we introduce goal classification and goal reconstruction tasks
and evaluation frameworks, and use large-scale eye tracking for reading data in
English with hundreds of text-specific information seeking tasks. We develop
and compare several discriminative and generative multimodal LLMs that combine
eye movements and text for goal classification and goal reconstruction. Our
experiments show considerable success on both tasks, suggesting that LLMs can
extract valuable information about the readers' text-specific goals from eye
movements.
LLMs can ascertain readers' specific information goals through eye movement data, as demonstrated by successful goal classification and reconstruction tasks.
LLMs eye movements goal classification goal reconstruction multimodal large-scale eye tracking