Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with
  Jigsaw Puzzles
The application of rule-based reinforcement learning (RL) to multimodal large
language models (MLLMs) introduces unique challenges and potential deviations
from findings in text-only domains, particularly for perception-heavy tasks.
This paper provides a comprehensive study of rule-based visual RL, using jigsaw
puzzles as a structured experimental framework. Jigsaw puzzles offer inherent
ground truth, adjustable difficulty, and demand complex decision-making, making
them ideal for this study. Our research reveals several key findings:
Firstly, we find that MLLMs, initially performing near to random
guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and
generalize to complex, unseen configurations through fine-tuning.
Secondly, training on jigsaw puzzles can induce generalization to
other visual tasks, with effectiveness tied to specific task configurations.
Thirdly, MLLMs can learn and generalize with or without explicit
reasoning, though open-source models often favor direct answering.
Consequently, even when trained for step-by-step reasoning, they can ignore the
thinking process in deriving the final answer. Fourthly, we observe
that complex reasoning patterns appear to be pre-existing rather than emergent,
with their frequency increasing alongside training and task difficulty.
Finally, our results demonstrate that RL exhibits more effective
generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start
phase can hinder subsequent RL optimization. Although these observations are
based on jigsaw puzzles and may vary across other visual tasks, this research
contributes a valuable piece of jigsaw to the larger puzzle of collective
understanding rule-based visual RL and its potential in multimodal learning.
The code is available at: https://github.com/zifuwanggg/Jigsaw-R1.
683e6b2d97fd742a8edb8a8e
Zifu Wang
admin_assigned
2025-06-03T09:41:52.137Z
64f5c7cb65a4b1acb20ffc15
/avatars/67c42a6e26bbeddc4bf706eb8f1a75b1.svg
Zifu Wang
user
wangzifu
683e6b2d97fd742a8edb8a8f
Junyi Zhu
admin_assigned
2025-06-03T09:42:00.703Z
6477b2038ab7e732b6d8a9b5
/avatars/c859a35b8965a904f103bbc34f36ab2a.svg
Junyi Zhu
user
JJZhu-AI
683e6b2d97fd742a8edb8a90
Bo Tang
683e6b2d97fd742a8edb8a91
Zhiyu Li
683e6b2d97fd742a8edb8a92
Feiyu Xiong
683e6b2d97fd742a8edb8a93
Jiaqian Yu
683e6b2d97fd742a8edb8a94
Matthew B. Blaschko
Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with
  Jigsaw Puzzles
The application of rule-based reinforcement learning (RL) to multimodal large
language models (MLLMs) introduces unique challenges and potential deviations
from findings in text-only domains, particularly for perception-heavy tasks.
This paper provides a comprehensive study of rule-based visual RL, using jigsaw
puzzles as a structured experimental framework. Jigsaw puzzles offer inherent
ground truth, adjustable difficulty, and demand complex decision-making, making
them ideal for this study. Our research reveals several key findings:
Firstly, we find that MLLMs, initially performing near to random
guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and
generalize to complex, unseen configurations through fine-tuning.
Secondly, training on jigsaw puzzles can induce generalization to
other visual tasks, with effectiveness tied to specific task configurations.
Thirdly, MLLMs can learn and generalize with or without explicit
reasoning, though open-source models often favor direct answering.
Consequently, even when trained for step-by-step reasoning, they can ignore the
thinking process in deriving the final answer. Fourthly, we observe
that complex reasoning patterns appear to be pre-existing rather than emergent,
with their frequency increasing alongside training and task difficulty.
Finally, our results demonstrate that RL exhibits more effective
generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start
phase can hinder subsequent RL optimization. Although these observations are
based on jigsaw puzzles and may vary across other visual tasks, this research
contributes a valuable piece of jigsaw to the larger puzzle of collective
understanding rule-based visual RL and its potential in multimodal learning.
The code is available at: https://github.com/zifuwanggg/Jigsaw-R1.
Rule-based reinforcement learning applied to multimodal large language models demonstrates effective generalization in visual tasks, particularly using jigsaw puzzles, outperforming supervised fine-tuning.
rule-based reinforcement learning multimodal large language models visual RL jigsaw puzzles fine-tuning supervised fine-tuning complex decision-making visual tasks step-by-step reasoning generalization