Learning on the Job: Test-Time Curricula for Targeted Reinforcement
  Learning
Humans are good at learning on the job: We learn how to solve the tasks we
face as we go along. Can a model do the same? We propose an agent that
assembles a task-specific curriculum, called test-time curriculum (TTC-RL), and
applies reinforcement learning to continue training the model for its target
task. The test-time curriculum avoids time-consuming human curation of datasets
by automatically selecting the most task-relevant data from a large pool of
available training data. Our experiments demonstrate that reinforcement
learning on a test-time curriculum consistently improves the model on its
target tasks, across a variety of evaluations and models. Notably, on
challenging math and coding benchmarks, TTC-RL improves the pass@1 of Qwen3-8B
by approximately 1.8x on AIME25 and 2.1x on CodeElo. Moreover, we find that
TTC-RL significantly raises the performance ceiling compared to the initial
model, increasing pass@8 on AIME25 from 40% to 62% and on CodeElo from 28% to
43%. Our findings show the potential of test-time curricula in extending the
test-time scaling paradigm to continual training on thousands of task-relevant
experiences during test-time.
68e4f3b2ee2331580b78244c
Jonas Hübotter
claimed_verified
2025-10-07T12:25:08.212Z
6645dbab80ee4eb5ddc8aed8
https://cdn-avatars.huggingface.co/v1/production/uploads/6645dbab80ee4eb5ddc8aed8/-IckaooWMXeACKew9THv_.jpeg
Jonas Hübotter
user
jonhue
68e4f3b2ee2331580b78244d
Leander Diaz-Bone
68e4f3b2ee2331580b78244e
Ido Hakimi
68e4f3b2ee2331580b78244f
Andreas Krause
68e4f3b2ee2331580b782450
Moritz Hardt
Learning on the Job: Test-Time Curricula for Targeted Reinforcement
  Learning
Humans are good at learning on the job: We learn how to solve the tasks we
face as we go along. Can a model do the same? We propose an agent that
assembles a task-specific curriculum, called test-time curriculum (TTC-RL), and
applies reinforcement learning to continue training the model for its target
task. The test-time curriculum avoids time-consuming human curation of datasets
by automatically selecting the most task-relevant data from a large pool of
available training data. Our experiments demonstrate that reinforcement
learning on a test-time curriculum consistently improves the model on its
target tasks, across a variety of evaluations and models. Notably, on
challenging math and coding benchmarks, TTC-RL improves the pass@1 of Qwen3-8B
by approximately 1.8x on AIME25 and 2.1x on CodeElo. Moreover, we find that
TTC-RL significantly raises the performance ceiling compared to the initial
model, increasing pass@8 on AIME25 from 40% to 62% and on CodeElo from 28% to
43%. Our findings show the potential of test-time curricula in extending the
test-time scaling paradigm to continual training on thousands of task-relevant
experiences during test-time.
Test-time curriculum (TTC-RL) uses reinforcement learning to dynamically select task-relevant data, improving model performance on challenging benchmarks without human curation.
test-time curriculum TTC-RL reinforcement learning task-specific curriculum pass@1 pass@8 AIME25 CodeElo continual training