Cache-to-Cache: Direct Semantic Communication Between Large Language
  Models
Multi-LLM systems harness the complementary strengths of diverse Large
Language Models, achieving performance and efficiency gains unattainable by a
single model. In existing designs, LLMs communicate through text, forcing
internal representations to be transformed into output token sequences. This
process both loses rich semantic information and incurs token-by-token
generation latency. Motivated by these limitations, we ask: Can LLMs
communicate beyond text? Oracle experiments show that enriching the KV-Cache
semantics can improve response quality without increasing cache size,
supporting KV-Cache as an effective medium for inter-model communication. Thus,
we propose Cache-to-Cache (C2C), a new paradigm for direct semantic
communication between LLMs. C2C uses a neural network to project and fuse the
source model's KV-cache with that of the target model to enable direct semantic
transfer. A learnable gating mechanism selects the target layers that benefit
from cache communication. Compared with text communication, C2C utilizes the
deep, specialized semantics from both models, while avoiding explicit
intermediate text generation. Experiments show that C2C achieves 8.5-10.5%
higher average accuracy than individual models. It further outperforms the text
communication paradigm by approximately 3.0-5.0%, while delivering an average
2.0x speedup in latency. Our code is available at
https://github.com/thu-nics/C2C.
68e7171b7ae125f9582e6952
Tianyu Fu
claimed_verified
2025-10-09T06:50:21.158Z
6445fd9ba56444c355dcbcba
https://cdn-avatars.huggingface.co/v1/production/uploads/6445fd9ba56444c355dcbcba/NCyRCD-MK-yA0_qY6I2y0.png
Tianyu Fu
user
fuvty
68e7171b7ae125f9582e6953
Zihan Min
claimed_verified
2025-10-09T12:48:53.564Z
6607b4a850d2b7a7109f0496
/avatars/ab52887b31b5ac8d20f91bf3b0db674d.svg
Zihan Min
user
minzh23
68e7171b7ae125f9582e6954
Hanling Zhang
claimed_verified
2025-10-15T07:08:11.348Z
6599c0078982abaa5cde36c8
/avatars/a94202b7f735eb43baa25e5ac0234be1.svg
Hanling Zhang
user
p2o6e100
68e7171b7ae125f9582e6955
Jichao Yan
claimed_verified
2025-10-09T12:48:51.583Z
6800abdd637c165b6c03c429
/avatars/739dbc76409cbef66528417710a9e7c6.svg
JICHAO Yan
user
elentsing
68e7171b7ae125f9582e6956
Guohao Dai
68e7171b7ae125f9582e6957
Wanli Ouyang
68e7171b7ae125f9582e6958
Yu Wang
Cache-to-Cache: Direct Semantic Communication Between Large Language
  Models
Multi-LLM systems harness the complementary strengths of diverse Large
Language Models, achieving performance and efficiency gains unattainable by a
single model. In existing designs, LLMs communicate through text, forcing
internal representations to be transformed into output token sequences. This
process both loses rich semantic information and incurs token-by-token
generation latency. Motivated by these limitations, we ask: Can LLMs
communicate beyond text? Oracle experiments show that enriching the KV-Cache
semantics can improve response quality without increasing cache size,
supporting KV-Cache as an effective medium for inter-model communication. Thus,
we propose Cache-to-Cache (C2C), a new paradigm for direct semantic
communication between LLMs. C2C uses a neural network to project and fuse the
source model's KV-cache with that of the target model to enable direct semantic
transfer. A learnable gating mechanism selects the target layers that benefit
from cache communication. Compared with text communication, C2C utilizes the
deep, specialized semantics from both models, while avoiding explicit
intermediate text generation. Experiments show that C2C achieves 8.5-10.5%
higher average accuracy than individual models. It further outperforms the text
communication paradigm by approximately 3.0-5.0%, while delivering an average
2.0x speedup in latency. Our code is available at
https://github.com/thu-nics/C2C.
Cache-to-Cache (C2C) enables direct semantic communication between LLMs using neural network projections, improving accuracy and reducing latency compared to text-based communication.
Large Language Models KV-Cache Cache-to-Cache neural network semantic communication gating mechanism accuracy latency