When Big Models Train Small Ones: Label-Free Model Parity Alignment for
  Efficient Visual Question Answering using Small VLMs
Large Vision-Language Models (L-VLMs) have demonstrated remarkable
performance in various vision and language tasks, including visual question
answering (VQA). However, their high computational cost makes them impractical
for resource-constrained settings and inference-heavy applications. In
contrast, Small Vision-Language Models (S-VLMs) offer efficiency but suffer
from a significant performance gap compared to their larger counterparts. In
this work, we introduce the Model Parity Aligner (MPA), a novel framework
designed to systematically improve S-VLMs by leveraging unlabeled images and
effective knowledge transfer from L-VLMs. Instead of traditional knowledge
distillation methods that rely on labeled training data, MPA employs a
strategic parity-based approach that precisely identifies the knowledge
disparities between S-VLMs and L-VLMs, and optimizes training by targeting only
these disparities. We conduct extensive experiments on four diverse VQA
benchmarks, namely TextVQA, ST-VQA, ChartQA, and OKVQA, each of which requires
specialized reasoning capabilities such as text recognition, chart
interpretation, and commonsense and factual understanding. Our results
demonstrate that MPA consistently enhances the performance of S-VLMs on all
benchmarks, reducing the performance gap while maintaining computational
efficiency. We make our code publicly available.
68d22a211ca7156988a8edee
Abhirama Subramanyam Penamakuri
claimed_verified
2025-09-23T10:05:37.522Z
6549d555fc80ab2d8ca1469a
https://cdn-avatars.huggingface.co/v1/production/uploads/6549d555fc80ab2d8ca1469a/Z6-tS-9-6-aUiHCAW5ql8.jpeg
Abhirama Subramanyam
user
abhiram4572
68d22a211ca7156988a8edef
Navlika Singh
admin_assigned
2025-09-23T14:35:35.942Z
62cc633787e44ff3e7119a25
/avatars/ec0230dfe0bc707e25f66cdb04e3a217.svg
Navlika Singh
user
NavlikaSingh
68d22a211ca7156988a8edf0
Piyush Arora
68d22a211ca7156988a8edf1
Anand Mishra
When Big Models Train Small Ones: Label-Free Model Parity Alignment for
  Efficient Visual Question Answering using Small VLMs
Large Vision-Language Models (L-VLMs) have demonstrated remarkable
performance in various vision and language tasks, including visual question
answering (VQA). However, their high computational cost makes them impractical
for resource-constrained settings and inference-heavy applications. In
contrast, Small Vision-Language Models (S-VLMs) offer efficiency but suffer
from a significant performance gap compared to their larger counterparts. In
this work, we introduce the Model Parity Aligner (MPA), a novel framework
designed to systematically improve S-VLMs by leveraging unlabeled images and
effective knowledge transfer from L-VLMs. Instead of traditional knowledge
distillation methods that rely on labeled training data, MPA employs a
strategic parity-based approach that precisely identifies the knowledge
disparities between S-VLMs and L-VLMs, and optimizes training by targeting only
these disparities. We conduct extensive experiments on four diverse VQA
benchmarks, namely TextVQA, ST-VQA, ChartQA, and OKVQA, each of which requires
specialized reasoning capabilities such as text recognition, chart
interpretation, and commonsense and factual understanding. Our results
demonstrate that MPA consistently enhances the performance of S-VLMs on all
benchmarks, reducing the performance gap while maintaining computational
efficiency. We make our code publicly available.
The Model Parity Aligner (MPA) framework improves Small Vision-Language Models (S-VLMs) by leveraging unlabeled images and knowledge transfer from Large Vision-Language Models (L-VLMs) to reduce performance gaps in vision and language tasks.
Large Vision-Language Models Small Vision-Language Models Model Parity Aligner knowledge transfer knowledge distillation VQA TextVQA ST-VQA ChartQA OKVQA specialized reasoning text recognition chart interpretation commonsense understanding factual understanding