DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning
Although Vision Language Models (VLMs) exhibit strong perceptual abilities
and impressive visual reasoning, they struggle with attention to detail and
precise action planning in complex, dynamic environments, leading to subpar
performance. Real-world tasks typically require complex interactions, advanced
spatial reasoning, long-term planning, and continuous strategy refinement,
usually necessitating understanding the physics rules of the target scenario.
However, evaluating these capabilities in real-world scenarios is often
prohibitively expensive. To bridge this gap, we introduce DeepPHY, a novel
benchmark framework designed to systematically evaluate VLMs' understanding and
reasoning about fundamental physical principles through a series of challenging
simulated environments. DeepPHY integrates multiple physical reasoning
environments of varying difficulty levels and incorporates fine-grained
evaluation metrics. Our evaluation finds that even state-of-the-art VLMs
struggle to translate descriptive physical knowledge into precise, predictive
control.
68956dc548b0ae5ca2710dc1
Xinrun Xu
claimed_verified
2025-08-11T06:50:15.359Z
653238fdcd5377e9adee0c41
/avatars/78aea70cde6ab0050c7e18b5e148075c.svg
Xinrun Xu
user
SherryXu
68956dc548b0ae5ca2710dc2
Pi Bu
68956dc548b0ae5ca2710dc3
Ye Wang
68956dc548b0ae5ca2710dc4
Börje F. Karlsson
claimed_verified
2025-08-08T16:19:28.764Z
61e52be53d6dbb1da842316a
https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg
Börje Karlsson
user
tellarin
68956dc548b0ae5ca2710dc5
Ziming Wang
68956dc548b0ae5ca2710dc6
Tengtao Song
68956dc548b0ae5ca2710dc7
Qi Zhu
68956dc548b0ae5ca2710dc8
Jun Song
68956dc548b0ae5ca2710dc9
Zhiming Ding
68956dc548b0ae5ca2710dca
Bo Zheng
DeepPHY: Benchmarking Agentic VLMs on Physical Reasoning
Although Vision Language Models (VLMs) exhibit strong perceptual abilities
and impressive visual reasoning, they struggle with attention to detail and
precise action planning in complex, dynamic environments, leading to subpar
performance. Real-world tasks typically require complex interactions, advanced
spatial reasoning, long-term planning, and continuous strategy refinement,
usually necessitating understanding the physics rules of the target scenario.
However, evaluating these capabilities in real-world scenarios is often
prohibitively expensive. To bridge this gap, we introduce DeepPHY, a novel
benchmark framework designed to systematically evaluate VLMs' understanding and
reasoning about fundamental physical principles through a series of challenging
simulated environments. DeepPHY integrates multiple physical reasoning
environments of varying difficulty levels and incorporates fine-grained
evaluation metrics. Our evaluation finds that even state-of-the-art VLMs
struggle to translate descriptive physical knowledge into precise, predictive
control.
DeepPHY evaluates Vision Language Models' physical reasoning and control through simulated environments with varying difficulty levels.
Vision Language Models DeepPHY physical reasoning simulated environments evaluation metrics