STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for
  Spoken Language Models
Spoken Language Models (SLMs) are designed to take speech inputs and produce
spoken responses. However, current SLMs lack the ability to perform an
internal, unspoken thinking process before responding. In contrast, humans
typically engage in complex mental reasoning internally, enabling them to
communicate ideas clearly and concisely. Thus, integrating an unspoken thought
process into SLMs is highly desirable. While naively generating a complete
chain-of-thought (CoT) reasoning before starting to talk can enable thinking
for SLMs, this induces additional latency for the speech response, as the CoT
reasoning can be arbitrarily long. To solve this issue, we propose Stitch, a
novel generation method that alternates between the generation of unspoken
reasoning chunks and spoken response chunks. Since the audio duration of a
chunk of spoken response is much longer than the time to generate the tokens in
a chunk of spoken response, we use the remaining free time to generate the
unspoken reasoning tokens. When a chunk of audio is played to the user, the
model continues to generate the next unspoken reasoning chunk, achieving
simultaneous thinking and talking. Remarkably, Stitch matches the latency of
baselines that cannot generate unspoken CoT by design while outperforming those
baselines by 15% on math reasoning datasets; Stitch also performs equally well
on non-reasoning datasets as those baseline models. Some animations and
demonstrations are on the project page: https://d223302.github.io/STITCH.
687f032533947f780d9b4b07
Cheng-Han Chiang
claimed_verified
2025-07-22T07:47:56.952Z
622326ae0129f2097d69a3e2
/avatars/7665223e3fc8b820ce001e6003daf4d2.svg
Cheng-Han Chiang
user
dcml0714
687f032533947f780d9b4b08
Xiaofei Wang
claimed_verified
2025-07-22T07:47:55.199Z
64dc191bc307ee5369fbcb04
/avatars/5a8a0db63a187e85d4ae2fff93a838f0.svg
Xiaofei Wang
user
xiaofei-wang
687f032533947f780d9b4b09
Linjie Li
687f032533947f780d9b4b0a
Chung-Ching Lin
claimed_verified
2025-07-23T08:37:17.483Z
6669f406d21cf7d39b1d98ba
/avatars/5ec270cb8bcb68786379d8a8bc411aaa.svg
CC Lin
user
cclin10
687f032533947f780d9b4b0b
Kevin Lin
687f032533947f780d9b4b0c
Shujie Liu
687f032533947f780d9b4b0d
Zhendong Wang
687f032533947f780d9b4b0e
Zhengyuan Yang
687f032533947f780d9b4b0f
Hung-yi Lee
687f032533947f780d9b4b10
Lijuan Wang
STITCH: Simultaneous Thinking and Talking with Chunked Reasoning for
  Spoken Language Models
Spoken Language Models (SLMs) are designed to take speech inputs and produce
spoken responses. However, current SLMs lack the ability to perform an
internal, unspoken thinking process before responding. In contrast, humans
typically engage in complex mental reasoning internally, enabling them to
communicate ideas clearly and concisely. Thus, integrating an unspoken thought
process into SLMs is highly desirable. While naively generating a complete
chain-of-thought (CoT) reasoning before starting to talk can enable thinking
for SLMs, this induces additional latency for the speech response, as the CoT
reasoning can be arbitrarily long. To solve this issue, we propose Stitch, a
novel generation method that alternates between the generation of unspoken
reasoning chunks and spoken response chunks. Since the audio duration of a
chunk of spoken response is much longer than the time to generate the tokens in
a chunk of spoken response, we use the remaining free time to generate the
unspoken reasoning tokens. When a chunk of audio is played to the user, the
model continues to generate the next unspoken reasoning chunk, achieving
simultaneous thinking and talking. Remarkably, Stitch matches the latency of
baselines that cannot generate unspoken CoT by design while outperforming those
baselines by 15% on math reasoning datasets; Stitch also performs equally well
on non-reasoning datasets as those baseline models. Some animations and
demonstrations are on the project page: https://d223302.github.io/STITCH.
Stitch is a novel method that enables simultaneous thinking and talking in spoken language models by alternating between generating unspoken reasoning and spoken response chunks, achieving low latency and improved performance on math reasoning tasks.
Spoken Language Models chain-of-thought unspoken reasoning spoken response audio duration tokens math reasoning datasets