OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn
  Dialogue with Large Language Models
As multi-turn dialogues with large language models (LLMs) grow longer and
more complex, how can users better evaluate and review progress on their
conversational goals? We present OnGoal, an LLM chat interface that helps users
better manage goal progress. OnGoal provides real-time feedback on goal
alignment through LLM-assisted evaluation, explanations for evaluation results
with examples, and overviews of goal progression over time, enabling users to
navigate complex dialogues more effectively. Through a study with 20
participants on a writing task, we evaluate OnGoal against a baseline chat
interface without goal tracking. Using OnGoal, participants spent less time and
effort to achieve their goals while exploring new prompting strategies to
overcome miscommunication, suggesting tracking and visualizing goals can
enhance engagement and resilience in LLM dialogues. Our findings inspired
design implications for future LLM chat interfaces that improve goal
communication, reduce cognitive load, enhance interactivity, and enable
feedback to improve LLM performance.
68b12b5eb19c5400014849c6
Adam Coscia
68b12b5eb19c5400014849c7
Shunan Guo
68b12b5eb19c5400014849c8
Eunyee Koh
68b12b5eb19c5400014849c9
Alex Endert
OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn
  Dialogue with Large Language Models
As multi-turn dialogues with large language models (LLMs) grow longer and
more complex, how can users better evaluate and review progress on their
conversational goals? We present OnGoal, an LLM chat interface that helps users
better manage goal progress. OnGoal provides real-time feedback on goal
alignment through LLM-assisted evaluation, explanations for evaluation results
with examples, and overviews of goal progression over time, enabling users to
navigate complex dialogues more effectively. Through a study with 20
participants on a writing task, we evaluate OnGoal against a baseline chat
interface without goal tracking. Using OnGoal, participants spent less time and
effort to achieve their goals while exploring new prompting strategies to
overcome miscommunication, suggesting tracking and visualizing goals can
enhance engagement and resilience in LLM dialogues. Our findings inspired
design implications for future LLM chat interfaces that improve goal
communication, reduce cognitive load, enhance interactivity, and enable
feedback to improve LLM performance.
OnGoal, an LLM chat interface with goal tracking, improves user efficiency and engagement in complex dialogues by providing real-time feedback and visualizations.
LLM chat interface goal alignment LLM-assisted evaluation goal progression writing task cognitive load interactivity feedback