Efficient Test-Time Scaling for Small Vision-Language Models
Small Vision-Language Models (VLMs) provide a computationally efficient
alternative to larger models, at the cost of weaker generalization abilities
and downstream task performance. These shortcomings could be addressed by
test-time scaling techniques, but existing methods are typically
computationally demanding, contradicting the resource-efficient design goals of
small models. To address these limitations, we propose two novel and efficient
test-time scaling strategies that leverage the model-internal features rather
than external supervision: (i) Test-Time Augmentation (TTAug), which generates
multiple augmented inputs and aggregates outputs at the token level without
parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model
parameters during inference using consensus-based pseudolabels from TTAug.
Through extensive experiments across nine benchmarks, we demonstrate consistent
performance improvements while maintaining computational efficiency suitable
for resource-constrained environments. The generality of our approach is
demonstrated both within models at different scales and across different VLMs
without additional tuning.
68e460e1e4e093a7044e4cae
Mehmet Onurcan Kaya
claimed_verified
2025-10-07T12:27:27.922Z
63be9021da08ed0544f36c38
/avatars/42511d8b1d3a3ef99bc154c98b72dfba.svg
onurcan
user
monurcan
68e460e1e4e093a7044e4caf
Desmond Elliott
68e460e1e4e093a7044e4cb0
Dim P. Papadopoulos
Efficient Test-Time Scaling for Small Vision-Language Models
Small Vision-Language Models (VLMs) provide a computationally efficient
alternative to larger models, at the cost of weaker generalization abilities
and downstream task performance. These shortcomings could be addressed by
test-time scaling techniques, but existing methods are typically
computationally demanding, contradicting the resource-efficient design goals of
small models. To address these limitations, we propose two novel and efficient
test-time scaling strategies that leverage the model-internal features rather
than external supervision: (i) Test-Time Augmentation (TTAug), which generates
multiple augmented inputs and aggregates outputs at the token level without
parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model
parameters during inference using consensus-based pseudolabels from TTAug.
Through extensive experiments across nine benchmarks, we demonstrate consistent
performance improvements while maintaining computational efficiency suitable
for resource-constrained environments. The generality of our approach is
demonstrated both within models at different scales and across different VLMs
without additional tuning.
Two novel test-time scaling strategies, Test-Time Augmentation and Test-Time Adaptation, improve small vision-language models' performance without compromising computational efficiency.
Test-Time Augmentation Test-Time Adaptation vision-language models token level parameter updates consensus-based pseudolabels resource-constrained environments