InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video
  Understanding
Modern multimodal large language models (MLLMs) can reason over hour-long
video, yet their key-value (KV) cache grows linearly with time--quickly
exceeding the fixed memory of phones, AR glasses, and edge robots. Prior
compression schemes either assume the whole video and user query are available
offline or must first build the full cache, so memory still scales with stream
length. InfiniPot-V is the first training-free, query-agnostic framework that
enforces a hard, length-independent memory cap for streaming video
understanding. During video encoding it monitors the cache and, once a user-set
threshold is reached, runs a lightweight compression pass that (i) removes
temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii)
keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four
open-source MLLMs and four long-video and two streaming-video benchmarks,
InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation,
and matches or surpasses full-cache accuracy--even in multi-turn dialogues. By
dissolving the KV cache bottleneck without retraining or query knowledge,
InfiniPot-V closes the gap for on-device streaming video assistants.
685911a30e4ad7e2197582f3
Minsoo Kim
claimed_verified
2025-06-24T08:09:40.953Z
63c0e2503bdc86f8108da51b
/avatars/7d47f11992f030b3d831e45102581d1f.svg
Minsoo Kim
user
minsoo2333
685911a30e4ad7e2197582f4
Kyuhong Shim
685911a30e4ad7e2197582f5
Jungwook Choi
685911a30e4ad7e2197582f6
Simyung Chang
InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video
  Understanding
Modern multimodal large language models (MLLMs) can reason over hour-long
video, yet their key-value (KV) cache grows linearly with time--quickly
exceeding the fixed memory of phones, AR glasses, and edge robots. Prior
compression schemes either assume the whole video and user query are available
offline or must first build the full cache, so memory still scales with stream
length. InfiniPot-V is the first training-free, query-agnostic framework that
enforces a hard, length-independent memory cap for streaming video
understanding. During video encoding it monitors the cache and, once a user-set
threshold is reached, runs a lightweight compression pass that (i) removes
temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii)
keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four
open-source MLLMs and four long-video and two streaming-video benchmarks,
InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation,
and matches or surpasses full-cache accuracy--even in multi-turn dialogues. By
dissolving the KV cache bottleneck without retraining or query knowledge,
InfiniPot-V closes the gap for on-device streaming video assistants.
InfiniPot-V is a training-free, query-agnostic framework that compresses the key-value cache during video encoding to maintain a fixed memory cap for streaming video understanding, enhancing real-time performance and accuracy.
multimodal large language models key-value cache Temporal-axis Redundancy Value-Norm ranking long-video benchmarks streaming-video benchmarks real-time generation multi-turn dialogues on-device streaming video assistants