FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal
  Distillation
Neural audio codecs are a fundamental component of modern generative audio
pipelines. Although recent codecs achieve strong low-bitrate reconstruction and
provide powerful representations for downstream tasks, most are non-streamable,
limiting their use in real-time applications. We present FocalCodec-Stream, a
hybrid codec based on focal modulation that compresses speech into a single
binary codebook at 0.55 - 0.80 kbps with a theoretical latency of 80 ms. Our
approach combines multi-stage causal distillation of WavLM with targeted
architectural improvements, including a lightweight refiner module that
enhances quality under latency constraints. Experiments show that
FocalCodec-Stream outperforms existing streamable codecs at comparable
bitrates, while preserving both semantic and acoustic information. The result
is a favorable trade-off between reconstruction quality, downstream task
performance, latency, and efficiency. Code and checkpoints will be released at
https://github.com/lucadellalib/focalcodec.
68d0a5f88adc5cd018d15a80
Luca Della Libera
claimed_verified
2025-09-22T03:01:09.815Z
63195d0582e7eec0eac040e3
https://cdn-avatars.huggingface.co/v1/production/uploads/63195d0582e7eec0eac040e3/0tXOYkMfmv9e53zBWgqz7.png
Luca Della Libera
user
lucadellalib
68d0a5f88adc5cd018d15a81
Cem Subakan
68d0a5f88adc5cd018d15a82
Mirco Ravanelli
FocalCodec-Stream: Streaming Low-Bitrate Speech Coding via Causal
  Distillation
Neural audio codecs are a fundamental component of modern generative audio
pipelines. Although recent codecs achieve strong low-bitrate reconstruction and
provide powerful representations for downstream tasks, most are non-streamable,
limiting their use in real-time applications. We present FocalCodec-Stream, a
hybrid codec based on focal modulation that compresses speech into a single
binary codebook at 0.55 - 0.80 kbps with a theoretical latency of 80 ms. Our
approach combines multi-stage causal distillation of WavLM with targeted
architectural improvements, including a lightweight refiner module that
enhances quality under latency constraints. Experiments show that
FocalCodec-Stream outperforms existing streamable codecs at comparable
bitrates, while preserving both semantic and acoustic information. The result
is a favorable trade-off between reconstruction quality, downstream task
performance, latency, and efficiency. Code and checkpoints will be released at
https://github.com/lucadellalib/focalcodec.
FocalCodec-Stream is a hybrid neural audio codec that achieves high-quality speech compression with low latency and is suitable for real-time applications.
neural audio codecs focal modulation binary codebook multi-stage causal distillation WavLM lightweight refiner module streamable codecs semantic information acoustic information reconstruction quality downstream task performance latency efficiency