OpenGVL - Benchmarking Visual Temporal Progress for Data Curation
Data scarcity remains one of the most limiting factors in driving progress in
robotics. However, the amount of available robotics data in the wild is growing
exponentially, creating new opportunities for large-scale data utilization.
Reliable temporal task completion prediction could help automatically annotate
and curate this data at scale. The Generative Value Learning (GVL) approach was
recently proposed, leveraging the knowledge embedded in vision-language models
(VLMs) to predict task progress from visual observations. Building upon GVL, we
propose OpenGVL, a comprehensive benchmark for estimating task progress across
diverse challenging manipulation tasks involving both robotic and human
embodiments. We evaluate the capabilities of publicly available open-source
foundation models, showing that open-source model families significantly
underperform closed-source counterparts, achieving only approximately 70% of
their performance on temporal progress prediction tasks. Furthermore, we
demonstrate how OpenGVL can serve as a practical tool for automated data
curation and filtering, enabling efficient quality assessment of large-scale
robotics datasets. We release the benchmark along with the complete codebase at
github.com/budzianowski/opengvl{OpenGVL}.
68d2cb3b0e215259d193b0db
Paweł Budzianowski
admin_assigned
2025-09-24T15:35:47.604Z
6273c47ef6d63a28483eaba2
/avatars/f50e8c37f070846814985091609aa23f.svg
Paweł Budzianowski
user
pfb30
68d2cb3b0e215259d193b0dc
Emilia Wiśnios
claimed_verified
2025-09-24T13:54:14.638Z
669fd3b34d95f13eb6e9dbd5
https://cdn-avatars.huggingface.co/v1/production/uploads/669fd3b34d95f13eb6e9dbd5/01VPhZO7bqgvCCaFFZbP3.jpeg
Emilia Wiśnios
user
emilia-wisnios
68d2cb3b0e215259d193b0dd
Gracjan Góral
68d2cb3b0e215259d193b0de
Igor Kulakov
68d2cb3b0e215259d193b0df
Viktor Petrenko
68d2cb3b0e215259d193b0e0
Krzysztof Walas
OpenGVL - Benchmarking Visual Temporal Progress for Data Curation
Data scarcity remains one of the most limiting factors in driving progress in
robotics. However, the amount of available robotics data in the wild is growing
exponentially, creating new opportunities for large-scale data utilization.
Reliable temporal task completion prediction could help automatically annotate
and curate this data at scale. The Generative Value Learning (GVL) approach was
recently proposed, leveraging the knowledge embedded in vision-language models
(VLMs) to predict task progress from visual observations. Building upon GVL, we
propose OpenGVL, a comprehensive benchmark for estimating task progress across
diverse challenging manipulation tasks involving both robotic and human
embodiments. We evaluate the capabilities of publicly available open-source
foundation models, showing that open-source model families significantly
underperform closed-source counterparts, achieving only approximately 70% of
their performance on temporal progress prediction tasks. Furthermore, we
demonstrate how OpenGVL can serve as a practical tool for automated data
curation and filtering, enabling efficient quality assessment of large-scale
robotics datasets. We release the benchmark along with the complete codebase at
github.com/budzianowski/opengvl{OpenGVL}.
OpenGVL is a benchmark for task progress prediction in robotics using vision-language models, showing open-source models underperform compared to closed-source ones and enabling automated data curation.
Generative Value Learning GVL vision-language models VLMs task progress prediction manipulation tasks open-source foundation models closed-source counterparts automated data curation quality assessment robotics datasets