VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery
Analyzing cultural-heritage artifacts remains challenging for MLLMs: general
models lack domain expertise, and SFT often overfits superficial patterns,
yielding brittle reasoning for authentication and historical attribution. This
raises the question of how to equip MLLMs with robust, expert-level reasoning
for ancient Greek pottery. We present VaseVL, an SFT-then-RL system that turns
evaluation into supervision: we construct a taxonomy of question types, probe
the SFT model to localize type-specific performance gaps, and optimize with
type-conditioned, compositionality-oriented rewards targeting those gaps. We
also release VaseVQA, a comprehensive benchmark of 31,773 images designed to
probe deep understanding. Experiments show state-of-the-art results on style
classification and historical attribution with marked gains in compositional
robustness over SFT-only baselines, validating diagnosis-guided,
taxonomy-conditioned reward engineering and providing a reusable resource for
future research. Code and dataset will be available at
https://github.com/AIGeeksGroup/VaseVQA.
68d2362b1ca7156988a8ee22
Jinchao Ge
admin_assigned
2025-09-23T14:28:57.036Z
671f28d2b84710ee860c97b1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/U-iDSeDjguCgfLkDRzYqR.png
C
user
JinchaoGe
68d2362b1ca7156988a8ee23
Tengfei Cheng
admin_assigned
2025-09-23T14:29:06.084Z
681890f3b6b878e74b799452
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/owBC2uR_qE00pLZT8nW8h.png
Tengfei Cheng
user
tengfeiCheng
68d2362b1ca7156988a8ee24
Biao Wu
68d2362b1ca7156988a8ee25
Zeyu Zhang
admin_assigned
2025-09-23T14:29:23.128Z
65b00730403a23a2fd765110
https://cdn-avatars.huggingface.co/v1/production/uploads/65b00730403a23a2fd765110/Uw-obs-VymyU9iMVKLURZ.jpeg
Zeyu Zhang
user
ZeyuZhang
68d2362b1ca7156988a8ee26
Shiya Huang
68d2362b1ca7156988a8ee27
Judith Bishop
68d2362b1ca7156988a8ee28
Gillian Shepherd
68d2362b1ca7156988a8ee29
Meng Fang
68d2362b1ca7156988a8ee2a
Ling Chen
68d2362b1ca7156988a8ee2b
Yang Zhao
VaseVQA: Multimodal Agent and Benchmark for Ancient Greek Pottery
Analyzing cultural-heritage artifacts remains challenging for MLLMs: general
models lack domain expertise, and SFT often overfits superficial patterns,
yielding brittle reasoning for authentication and historical attribution. This
raises the question of how to equip MLLMs with robust, expert-level reasoning
for ancient Greek pottery. We present VaseVL, an SFT-then-RL system that turns
evaluation into supervision: we construct a taxonomy of question types, probe
the SFT model to localize type-specific performance gaps, and optimize with
type-conditioned, compositionality-oriented rewards targeting those gaps. We
also release VaseVQA, a comprehensive benchmark of 31,773 images designed to
probe deep understanding. Experiments show state-of-the-art results on style
classification and historical attribution with marked gains in compositional
robustness over SFT-only baselines, validating diagnosis-guided,
taxonomy-conditioned reward engineering and providing a reusable resource for
future research. Code and dataset will be available at
https://github.com/AIGeeksGroup/VaseVQA.
VaseVL, an SFT-then-RL system, enhances MLLMs for ancient Greek pottery analysis by addressing performance gaps through taxonomy-conditioned rewards, achieving state-of-the-art results in style classification and historical attribution.
SFT RL VaseVL taxonomy type-specific performance gaps type-conditioned compositionality-oriented rewards VaseVQA compositional robustness