Thyme: Think Beyond Images
Following OpenAI's introduction of the ``thinking with images'' concept,
recent efforts have explored stimulating the use of visual information in the
reasoning process to enhance model performance in perception and reasoning
tasks. However, to the best of our knowledge, no open-source work currently
offers a feature set as rich as proprietary models (O3), which can perform
diverse image manipulations and simultaneously enhance logical reasoning
capabilities through code. In this paper, we make a preliminary attempt in this
direction by introducing Thyme (Think Beyond Images), a novel paradigm for
enabling MLLMs to transcend existing ``think with images'' approaches by
autonomously generating and executing diverse image processing and
computational operations via executable code. This approach not only
facilitates a rich, on-the-fly set of image manipulations (e.g., cropping,
rotation, contrast enhancement) but also allows for mathematical computations,
all while maintaining high autonomy in deciding when and how to apply these
operations. We activate this capability through a two-stage training strategy:
an initial SFT on a curated dataset of 500K samples to teach code generation,
followed by a RL phase to refine decision-making. For the RL stage, we manually
collect and design high-resolution question-answer pairs to increase the
learning difficulty, and we propose GRPO-ATS (Group Relative Policy
Optimization with Adaptive Temperature Sampling), an algorithm that applies
distinct temperatures to text and code generation to balance reasoning
exploration with code execution precision. We conduct extensive experimental
analysis and ablation studies. Comprehensive evaluations on nearly 20
benchmarks show that Thyme yields significant and consistent performance gains,
particularly in challenging high-resolution perception and complex reasoning
tasks.
68a286afa4caabb4320e63b6
Yi-Fan Zhang
claimed_verified
2025-08-19T15:58:45.502Z
623d8ca4c29adf5ef6175615
https://cdn-avatars.huggingface.co/v1/production/uploads/623d8ca4c29adf5ef6175615/q7lHao7UPwU1u7YLSP56m.jpeg
Yi-Fan Zhang
user
yifanzhang114
68a286afa4caabb4320e63b7
Xingyu Lu
claimed_verified
2025-08-19T10:49:18.830Z
673597cfc2424474d12ca58c
/avatars/f748f19619b07838a66bc419a7a6db9d.svg
xingyulu
user
Xingyulu47
68a286afa4caabb4320e63b8
Shukang Yin
68a286afa4caabb4320e63b9
Chaoyou Fu
68a286afa4caabb4320e63ba
Wei Chen
68a286afa4caabb4320e63bb
Xiao Hu
68a286afa4caabb4320e63bc
Bin Wen
68a286afa4caabb4320e63bd
Kaiyu Jiang
68a286afa4caabb4320e63be
Changyi Liu
68a286afa4caabb4320e63bf
Tianke Zhang
68a286afa4caabb4320e63c0
Haonan Fan
68a286afa4caabb4320e63c1
Kaibing Chen
68a286afa4caabb4320e63c2
Jiankang Chen
68a286afa4caabb4320e63c3
Haojie Ding
68a286afa4caabb4320e63c4
Kaiyu Tang
68a286afa4caabb4320e63c5
Zhang Zhang
68a286afa4caabb4320e63c6
Liang Wang
68a286afa4caabb4320e63c7
Fan Yang
68a286afa4caabb4320e63c8
Tingting Gao
claimed_verified
2025-09-03T08:30:06.725Z
68652063e29f1407b58da60f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/hTgS65zsKRPxELjSMDSNm.png
tingting gao
user
TinaGao
68a286afa4caabb4320e63c9
Guorui Zhou
Thyme: Think Beyond Images
Following OpenAI's introduction of the ``thinking with images'' concept,
recent efforts have explored stimulating the use of visual information in the
reasoning process to enhance model performance in perception and reasoning
tasks. However, to the best of our knowledge, no open-source work currently
offers a feature set as rich as proprietary models (O3), which can perform
diverse image manipulations and simultaneously enhance logical reasoning
capabilities through code. In this paper, we make a preliminary attempt in this
direction by introducing Thyme (Think Beyond Images), a novel paradigm for
enabling MLLMs to transcend existing ``think with images'' approaches by
autonomously generating and executing diverse image processing and
computational operations via executable code. This approach not only
facilitates a rich, on-the-fly set of image manipulations (e.g., cropping,
rotation, contrast enhancement) but also allows for mathematical computations,
all while maintaining high autonomy in deciding when and how to apply these
operations. We activate this capability through a two-stage training strategy:
an initial SFT on a curated dataset of 500K samples to teach code generation,
followed by a RL phase to refine decision-making. For the RL stage, we manually
collect and design high-resolution question-answer pairs to increase the
learning difficulty, and we propose GRPO-ATS (Group Relative Policy
Optimization with Adaptive Temperature Sampling), an algorithm that applies
distinct temperatures to text and code generation to balance reasoning
exploration with code execution precision. We conduct extensive experimental
analysis and ablation studies. Comprehensive evaluations on nearly 20
benchmarks show that Thyme yields significant and consistent performance gains,
particularly in challenging high-resolution perception and complex reasoning
tasks.
Thyme, a novel paradigm, enables MLLMs to autonomously perform image manipulations and computations, enhancing performance in perception and reasoning tasks through a two-stage training strategy and GRPO-ATS algorithm.
MLLMs think with images image processing computational operations executable code SFT RL GRPO-ATS Group Relative Policy Optimization Adaptive Temperature Sampling