LLAMAPIE: Proactive In-Ear Conversation Assistants
We introduce LlamaPIE, the first real-time proactive assistant designed to
enhance human conversations through discreet, concise guidance delivered via
hearable devices. Unlike traditional language models that require explicit user
invocation, this assistant operates in the background, anticipating user needs
without interrupting conversations. We address several challenges, including
determining when to respond, crafting concise responses that enhance
conversations, leveraging knowledge of the user for context-aware assistance,
and real-time, on-device processing. To achieve this, we construct a
semi-synthetic dialogue dataset and propose a two-model pipeline: a small model
that decides when to respond and a larger model that generates the response. We
evaluate our approach on real-world datasets, demonstrating its effectiveness
in providing helpful, unobtrusive assistance. User studies with our assistant,
implemented on Apple Silicon M2 hardware, show a strong preference for the
proactive assistant over both a baseline with no assistance and a reactive
model, highlighting the potential of LlamaPie to enhance live conversations.
68239705889f2a9d374c1774
Tuochao Chen
admin_assigned
2025-05-14T12:57:14.833Z
65a9d16f3b9e1f0f307c96c3
/avatars/882a0746a0adb2d7befe563f7fbc6321.svg
Tuochao Chen
user
tuochao
68239705889f2a9d374c1775
Nicholas Batchelder
admin_assigned
2025-05-14T12:57:20.805Z
64de5c6892f3bd97865b0e1e
/avatars/3141199aa7e1d345ec647af749eeff37.svg
Nicholas Batchelder
user
nicbat
68239705889f2a9d374c1776
Alisa Liu
68239705889f2a9d374c1777
Noah Smith
68239705889f2a9d374c1778
Shyamnath Gollakota
LLAMAPIE: Proactive In-Ear Conversation Assistants
We introduce LlamaPIE, the first real-time proactive assistant designed to
enhance human conversations through discreet, concise guidance delivered via
hearable devices. Unlike traditional language models that require explicit user
invocation, this assistant operates in the background, anticipating user needs
without interrupting conversations. We address several challenges, including
determining when to respond, crafting concise responses that enhance
conversations, leveraging knowledge of the user for context-aware assistance,
and real-time, on-device processing. To achieve this, we construct a
semi-synthetic dialogue dataset and propose a two-model pipeline: a small model
that decides when to respond and a larger model that generates the response. We
evaluate our approach on real-world datasets, demonstrating its effectiveness
in providing helpful, unobtrusive assistance. User studies with our assistant,
implemented on Apple Silicon M2 hardware, show a strong preference for the
proactive assistant over both a baseline with no assistance and a reactive
model, highlighting the potential of LlamaPie to enhance live conversations.
LlamaPIE, a real-time proactive assistant for conversations, uses a two-model pipeline with a semi-synthetic dialogue dataset to provide unobtrusive guidance via hearable devices.
hearable devices dialogue dataset two-model pipeline