Tina: Tiny Reasoning Models via LoRA
How cost-effectively can strong reasoning abilities be achieved in language
models? Driven by this fundamental question, we present Tina, a family of tiny
reasoning models achieved with high cost-efficiency. Notably, Tina demonstrates
that substantial reasoning performance can be developed using only minimal
resources, by applying parameter-efficient updates during reinforcement
learning (RL), using low-rank adaptation (LoRA), to an already tiny 1.5B
parameter base model. This minimalist approach produces models that achieve
reasoning performance which is competitive with, and sometimes surpasses, SOTA
RL reasoning models built upon the same base model. Crucially, this is achieved
at a tiny fraction of the computational post-training cost employed by existing
SOTA models. In fact, the best Tina model achieves a >20\% reasoning
performance increase and 43.33\% Pass@1 accuracy on AIME24, at only \$9 USD
post-training and evaluation cost (i.e., an estimated 260x cost reduction). Our
work reveals the surprising effectiveness of efficient RL reasoning via LoRA.
We validate this across multiple open-source reasoning datasets and various
ablation settings starting with a single, fixed set of hyperparameters.
Furthermore, we hypothesize that this effectiveness and efficiency stem from
LoRA rapidly adapting the model to the structural format of reasoning rewarded
by RL, while largely preserving the base model's underlying knowledge. In
service of accessibility and open research, we fully open-source all code,
training logs, and model weights \& checkpoints.
6808452d16c1c427ac727816
Shangshang Wang
claimed_verified
2025-04-24T09:11:10.259Z
67469d6a8407f929491dce06
https://cdn-avatars.huggingface.co/v1/production/uploads/67469d6a8407f929491dce06/7vd7zyApmT4rXe58gqmG1.png
Shangshang Wang
user
upup-ashton-wang
6808452d16c1c427ac727817
Julian Asilis
claimed_verified
2025-04-25T08:35:32.401Z
66b267189f7af7b98677f1cd
https://cdn-avatars.huggingface.co/v1/production/uploads/66b267189f7af7b98677f1cd/B9zsSH-ZVOdlp9y9ngdWO.png
Julian Asilis
user
asil012
6808452d16c1c427ac727818
Ömer Faruk Akgül
claimed_verified
2025-04-25T08:35:28.744Z
6425e8f7ba51f8a2136770d7
/avatars/86c96f061e574c6448d9526c52abd15a.svg
Omer Faruk Akgul
user
farukakgul
6808452d16c1c427ac727819
Enes Burak Bilgin
claimed_verified
2025-04-25T08:35:26.455Z
67410bd671fd1b3da6ca44ce
/avatars/56aa9a494bb6ee34ce3a26625f0b1f92.svg
Enes Burak Bilgin
user
ebbilgin
6808452d16c1c427ac72781a
Ollie Liu
6808452d16c1c427ac72781b
Willie Neiswanger
claimed_verified
2025-04-25T08:35:24.290Z
644bf65522d211df6444a7f4
https://cdn-avatars.huggingface.co/v1/production/uploads/644bf65522d211df6444a7f4/k_ddZdQDg2fzhwjI1EXyx.jpeg
Willie Neiswanger
user
willieneis
Tina: Tiny Reasoning Models via LoRA
How cost-effectively can strong reasoning abilities be achieved in language
models? Driven by this fundamental question, we present Tina, a family of tiny
reasoning models achieved with high cost-efficiency. Notably, Tina demonstrates
that substantial reasoning performance can be developed using only minimal
resources, by applying parameter-efficient updates during reinforcement
learning (RL), using low-rank adaptation (LoRA), to an already tiny 1.5B
parameter base model. This minimalist approach produces models that achieve
reasoning performance which is competitive with, and sometimes surpasses, SOTA
RL reasoning models built upon the same base model. Crucially, this is achieved
at a tiny fraction of the computational post-training cost employed by existing
SOTA models. In fact, the best Tina model achieves a >20\% reasoning
performance increase and 43.33\% Pass@1 accuracy on AIME24, at only \$9 USD
post-training and evaluation cost (i.e., an estimated 260x cost reduction). Our
work reveals the surprising effectiveness of efficient RL reasoning via LoRA.
We validate this across multiple open-source reasoning datasets and various
ablation settings starting with a single, fixed set of hyperparameters.
Furthermore, we hypothesize that this effectiveness and efficiency stem from
LoRA rapidly adapting the model to the structural format of reasoning rewarded
by RL, while largely preserving the base model's underlying knowledge. In
service of accessibility and open research, we fully open-source all code,
training logs, and model weights \& checkpoints.
Tina, a family of tiny reasoning models, achieves high reasoning performance at minimal computational cost through LoRA during reinforcement learning.
reinforcement learning LoRA parameter-efficient updates