VLM in a flash: Neuron Chunking for Flash-offloaded Vision-Language Model Inference
Kichang Yang, Seonjun Kim, Minjae Kim, Nairan Zhang, Chi Zhang, Youngki Lee
Edge deployment of large vision-language models (VLMs) increasingly relies on flash-based weight offloading, where activation sparsification is used to reduce I/O overhead. However, conventional sparsification techniques are model-centric—they select neurons based solely on activation magnitude, ignoring flash storage behavior. In this paper, we propose \textsc{Neuron Chunking}, a structured sparsification method that jointly considers activation importance and flash I/O efficiency. We introduce a contiguity-based latency model that accurately predicts the cost of arbitrary access patterns, and a multi-scale chunk selection algorithm that favors contiguous chunks with a high importance-to-latency ratio. Our approach significantly improves I/O efficiency, achieving up to 4.58× and 5.85× speedups over existing sparsification methods on Jetson Orin Nano and Jetson AGX Orin, respectively.