Towards a Personal Health Large Language Model
In health, most large language model (LLM) research has focused on clinical
tasks. However, mobile and wearable devices, which are rarely integrated into
such tasks, provide rich, longitudinal data for personal health monitoring.
Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from
Gemini for understanding and reasoning over numerical time-series personal
health data. We created and curated three datasets that test 1) production of
personalized insights and recommendations from sleep patterns, physical
activity, and physiological responses, 2) expert domain knowledge, and 3)
prediction of self-reported sleep outcomes. For the first task we designed 857
case studies in collaboration with domain experts to assess real-world
scenarios in sleep and fitness. Through comprehensive evaluation of
domain-specific rubrics, we observed that Gemini Ultra 1.0 and PH-LLM are not
statistically different from expert performance in fitness and, while experts
remain superior for sleep, fine-tuning PH-LLM provided significant improvements
in using relevant domain knowledge and personalizing information for sleep
insights. We evaluated PH-LLM domain knowledge using multiple choice sleep
medicine and fitness examinations. PH-LLM achieved 79% on sleep and 88% on
fitness, exceeding average scores from a sample of human experts. Finally, we
trained PH-LLM to predict self-reported sleep quality outcomes from textual and
multimodal encoding representations of wearable data, and demonstrate that
multimodal encoding is required to match performance of specialized
discriminative models. Although further development and evaluation are
necessary in the safety-critical personal health domain, these results
demonstrate both the broad knowledge and capabilities of Gemini models and the
benefit of contextualizing physiological data for personal health applications
as done with PH-LLM.
6667c6690c21d689e19101e7
Justin Cosentino
6667c6690c21d689e19101e8
Anastasiya Belyaeva
6667c6690c21d689e19101e9
Xin Liu
6667c6690c21d689e19101ea
Nicholas A. Furlotte
6667c6690c21d689e19101eb
Zhun Yang
6667c6690c21d689e19101ec
Chace Lee
6667c6690c21d689e19101ed
Erik Schenck
6667c6690c21d689e19101ee
Yojan Patel
6667c6690c21d689e19101ef
Jian Cui
6667c6690c21d689e19101f0
Logan Douglas Schneider
6667c6690c21d689e19101f1
Robby Bryant
6667c6690c21d689e19101f2
Ryan G. Gomes
6667c6690c21d689e19101f3
Allen Jiang
6667c6690c21d689e19101f4
Roy Lee
6667c6690c21d689e19101f5
Yun Liu
6667c6690c21d689e19101f6
Javier Perez
6667c6690c21d689e19101f7
Jameson K. Rogers
6667c6690c21d689e19101f8
Cathy Speed
6667c6690c21d689e19101f9
Shyam Tailor
6667c6690c21d689e19101fa
Megan Walker
6667c6690c21d689e19101fb
Jeffrey Yu
6667c6690c21d689e19101fc
Tim Althoff
6667c6690c21d689e19101fd
Conor Heneghan
6667c6690c21d689e19101fe
John Hernandez
6667c6690c21d689e19101ff
Mark Malhotra
6667c6690c21d689e1910200
Leor Stern
6667c6690c21d689e1910201
Yossi Matias
6667c6690c21d689e1910202
Greg S. Corrado
6667c6690c21d689e1910203
Shwetak Patel
6667c6690c21d689e1910204
Shravya Shetty
6667c6690c21d689e1910205
Jiening Zhan
6667c6690c21d689e1910206
Shruthi Prabhakara
6667c6690c21d689e1910207
Daniel McDuff
6667c6690c21d689e1910208
Cory Y. McLean
Towards a Personal Health Large Language Model
In health, most large language model (LLM) research has focused on clinical
tasks. However, mobile and wearable devices, which are rarely integrated into
such tasks, provide rich, longitudinal data for personal health monitoring.
Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from
Gemini for understanding and reasoning over numerical time-series personal
health data. We created and curated three datasets that test 1) production of
personalized insights and recommendations from sleep patterns, physical
activity, and physiological responses, 2) expert domain knowledge, and 3)
prediction of self-reported sleep outcomes. For the first task we designed 857
case studies in collaboration with domain experts to assess real-world
scenarios in sleep and fitness. Through comprehensive evaluation of
domain-specific rubrics, we observed that Gemini Ultra 1.0 and PH-LLM are not
statistically different from expert performance in fitness and, while experts
remain superior for sleep, fine-tuning PH-LLM provided significant improvements
in using relevant domain knowledge and personalizing information for sleep
insights. We evaluated PH-LLM domain knowledge using multiple choice sleep
medicine and fitness examinations. PH-LLM achieved 79% on sleep and 88% on
fitness, exceeding average scores from a sample of human experts. Finally, we
trained PH-LLM to predict self-reported sleep quality outcomes from textual and
multimodal encoding representations of wearable data, and demonstrate that
multimodal encoding is required to match performance of specialized
discriminative models. Although further development and evaluation are
necessary in the safety-critical personal health domain, these results
demonstrate both the broad knowledge and capabilities of Gemini models and the
benefit of contextualizing physiological data for personal health applications
as done with PH-LLM.
A Personal Health Large Language Model (PH-LLM) fine-tuned from Gemini demonstrates substantial capabilities in processing and interpreting personal health data, offering personalized insights and outperforming experts in certain domain-specific tasks.
Large Language Model (LLM) Personal Health Large Language Model (PH-LLM) Gemini fine-tuned numerical time-series data personalized insights recommendations sleep patterns physical activity physiological responses domain-specific rubrics expert performance multiple choice examinations multimodal encoding sleep quality outcomes specialized discriminative models contextualizing physiological data