OpenTSLM: Time-Series Language Models for Reasoning over Multivariate
  Medical Text- and Time-Series Data
LLMs have emerged as powerful tools for interpreting multimodal data. In
medicine, they hold particular promise for synthesizing large volumes of
clinical information into actionable insights and digital health applications.
Yet, a major limitation remains their inability to handle time series. To
overcome this gap, we present OpenTSLM, a family of Time Series Language Models
(TSLMs) created by integrating time series as a native modality to pretrained
LLMs, enabling reasoning over multiple time series of any length. We
investigate two architectures for OpenTSLM. The first, OpenTSLM-SoftPrompt,
models time series implicitly by concatenating learnable time series tokens
with text tokens via soft prompting. Although parameter-efficient, we
hypothesize that explicit time series modeling scales better and outperforms
implicit approaches. We thus introduce OpenTSLM-Flamingo, which integrates time
series with text via cross-attention. We benchmark both variants against
baselines that treat time series as text tokens or plots, across a suite of
text-time-series Chain-of-Thought (CoT) reasoning tasks. We introduce three
datasets: HAR-CoT, Sleep-CoT, and ECG-QA-CoT. Across all, OpenTSLM models
outperform baselines, reaching 69.9 F1 in sleep staging and 65.4 in HAR,
compared to 9.05 and 52.2 for finetuned text-only models. Notably, even
1B-parameter OpenTSLM models surpass GPT-4o (15.47 and 2.95). OpenTSLM-Flamingo
matches OpenTSLM-SoftPrompt in performance and outperforms on longer sequences,
while maintaining stable memory requirements. By contrast, SoftPrompt grows
exponentially in memory with sequence length, requiring around 110 GB compared
to 40 GB VRAM when training on ECG-QA with LLaMA-3B. Expert reviews by
clinicians find strong reasoning capabilities exhibited by OpenTSLMs on ECG-QA.
To facilitate further research, we provide all code, datasets, and models
open-source.
68e433d3e4e093a7044e4c49
Patrick Langer
claimed_verified
2025-10-07T12:27:35.380Z
68014c663a3230f390598c08
/avatars/93b74544e75d7f531138f66ff224f5c6.svg
Patrick
user
RealLast
68e433d3e4e093a7044e4c4a
Thomas Kaar
claimed_verified
2025-10-07T12:27:32.692Z
68019ed368af2860f16139ff
/avatars/951aad00ae4c2339f3ffe5bf8db07246.svg
Thomas Kaar
user
tkaar
68e433d3e4e093a7044e4c4b
Max Rosenblattl
68e433d3e4e093a7044e4c4c
Maxwell A. Xu
68e433d3e4e093a7044e4c4d
Winnie Chow
68e433d3e4e093a7044e4c4e
Martin Maritsch
68e433d3e4e093a7044e4c4f
Aradhana Verma
68e433d3e4e093a7044e4c50
Brian Han
68e433d3e4e093a7044e4c51
Daniel Seung Kim
68e433d3e4e093a7044e4c52
Henry Chubb
68e433d3e4e093a7044e4c53
Scott Ceresnak
68e433d3e4e093a7044e4c54
Aydin Zahedivash
68e433d3e4e093a7044e4c55
Alexander Tarlochan Singh Sandhu
68e433d3e4e093a7044e4c56
Fatima Rodriguez
68e433d3e4e093a7044e4c57
Daniel McDuff
68e433d3e4e093a7044e4c58
Elgar Fleisch
68e433d3e4e093a7044e4c59
Oliver Aalami
68e433d3e4e093a7044e4c5a
Filipe Barata
68e433d3e4e093a7044e4c5b
Paul Schmiedmayer
OpenTSLM: Time-Series Language Models for Reasoning over Multivariate
  Medical Text- and Time-Series Data
LLMs have emerged as powerful tools for interpreting multimodal data. In
medicine, they hold particular promise for synthesizing large volumes of
clinical information into actionable insights and digital health applications.
Yet, a major limitation remains their inability to handle time series. To
overcome this gap, we present OpenTSLM, a family of Time Series Language Models
(TSLMs) created by integrating time series as a native modality to pretrained
LLMs, enabling reasoning over multiple time series of any length. We
investigate two architectures for OpenTSLM. The first, OpenTSLM-SoftPrompt,
models time series implicitly by concatenating learnable time series tokens
with text tokens via soft prompting. Although parameter-efficient, we
hypothesize that explicit time series modeling scales better and outperforms
implicit approaches. We thus introduce OpenTSLM-Flamingo, which integrates time
series with text via cross-attention. We benchmark both variants against
baselines that treat time series as text tokens or plots, across a suite of
text-time-series Chain-of-Thought (CoT) reasoning tasks. We introduce three
datasets: HAR-CoT, Sleep-CoT, and ECG-QA-CoT. Across all, OpenTSLM models
outperform baselines, reaching 69.9 F1 in sleep staging and 65.4 in HAR,
compared to 9.05 and 52.2 for finetuned text-only models. Notably, even
1B-parameter OpenTSLM models surpass GPT-4o (15.47 and 2.95). OpenTSLM-Flamingo
matches OpenTSLM-SoftPrompt in performance and outperforms on longer sequences,
while maintaining stable memory requirements. By contrast, SoftPrompt grows
exponentially in memory with sequence length, requiring around 110 GB compared
to 40 GB VRAM when training on ECG-QA with LLaMA-3B. Expert reviews by
clinicians find strong reasoning capabilities exhibited by OpenTSLMs on ECG-QA.
To facilitate further research, we provide all code, datasets, and models
open-source.
OpenTSLM integrates time series into pretrained LLMs using soft prompting and cross-attention, outperforming text-only models on clinical reasoning tasks.
Time Series Language Models TSLMs soft prompting cross-attention Chain-of-Thought HAR-CoT Sleep-CoT ECG-QA-CoT F1 score GPT-4o LLaMA-3B