Less is More: Recursive Reasoning with Tiny Networks
Hierarchical Reasoning Model (HRM) is a novel approach using two small neural
networks recursing at different frequencies. This biologically inspired method
beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze,
and ARC-AGI while trained with small models (27M parameters) on small data
(around 1000 examples). HRM holds great promise for solving hard problems with
small networks, but it is not yet well understood and may be suboptimal. We
propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach
that achieves significantly higher generalization than HRM, while using a
single tiny network with only 2 layers. With only 7M parameters, TRM obtains
45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs
(e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the
parameters.
68e527e27e1f41b92bf8e9f9
Alexia Jolicoeur-Martineau
admin_assigned
2025-10-14T14:26:19.438Z
62f8ea1177b722f186611e8e
https://cdn-avatars.huggingface.co/v1/production/uploads/1660479589894-noauth.jpeg
Alexia Jolicoeur-Martineau
user
AlexiaJM
Less is More: Recursive Reasoning with Tiny Networks
Hierarchical Reasoning Model (HRM) is a novel approach using two small neural
networks recursing at different frequencies. This biologically inspired method
beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze,
and ARC-AGI while trained with small models (27M parameters) on small data
(around 1000 examples). HRM holds great promise for solving hard problems with
small networks, but it is not yet well understood and may be suboptimal. We
propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach
that achieves significantly higher generalization than HRM, while using a
single tiny network with only 2 layers. With only 7M parameters, TRM obtains
45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs
(e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the
parameters.
Tiny Recursive Model (TRM) achieves high generalization on complex puzzle tasks using a small, two-layer network with minimal parameters, outperforming larger language models.
Hierarchical Reasoning Model HRM Tiny Recursive Model TRM recursive reasoning neural networks ARC-AGI Deepseek R1 o3-mini Gemini 2.5 Pro