InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via
  Rubric-Based Incremental Training
Large Language Models (LLMs) have shown substantial advances through
reinforcement learning (RL), particularly in domains where rewards can be
programmatically verified, such as mathematics and code. In these areas, models
benefit from a well-defined operational base guided by explicit rule-based
objectives. However, this progress reveals a significant limitation: in
open-ended domains where rewards are ambiguous, subjective, or
context-dependent, such as creative writing, scientific reasoning, and notably
medical consultation, robust reward functions are lacking, making these areas
challenging for current RL strategies. To bridge this gap, we introduce ORBIT,
an open-ended rubric-based incremental training framework specifically designed
for high-stakes medical dialogue. ORBIT integrates syn- thetic dialogue
generation with the dynamic creation of rubrics, employing these rubrics to
direct an incremental RL process. In particular, this approach does not depend
on external medical knowledge or manual rules, instead utilizing rubric-guided
feedback to shape learning. When implemented on the Qwen3-4B-Instruct model,
our method can greatly enhance its performance on the HealthBench-Hard
benchmark from 7.0 to 27.2 using only 2k samples, thus achieving
state-of-the-art results for models of this scale. Our analysis confirms that
rubric-driven RL fos-ters consistent performance gains across diverse
consultation scenarios, going beyond simple numerical improvements. These
findings underscore rubric-based feedback as a scalable strategy for advancing
LLMs in intricate, open-ended tasks.
68f5bdca8589920bf4d3219b
Pengkai Wang
68f5bdca8589920bf4d3219c
Qi Zuo
68f5bdca8589920bf4d3219d
Pengwei Liu
68f5bdca8589920bf4d3219e
Zhijie Sang
68f5bdca8589920bf4d3219f
Congkai Xie
68f5bdca8589920bf4d321a0
Hongxia Yang
InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via
  Rubric-Based Incremental Training
Large Language Models (LLMs) have shown substantial advances through
reinforcement learning (RL), particularly in domains where rewards can be
programmatically verified, such as mathematics and code. In these areas, models
benefit from a well-defined operational base guided by explicit rule-based
objectives. However, this progress reveals a significant limitation: in
open-ended domains where rewards are ambiguous, subjective, or
context-dependent, such as creative writing, scientific reasoning, and notably
medical consultation, robust reward functions are lacking, making these areas
challenging for current RL strategies. To bridge this gap, we introduce ORBIT,
an open-ended rubric-based incremental training framework specifically designed
for high-stakes medical dialogue. ORBIT integrates syn- thetic dialogue
generation with the dynamic creation of rubrics, employing these rubrics to
direct an incremental RL process. In particular, this approach does not depend
on external medical knowledge or manual rules, instead utilizing rubric-guided
feedback to shape learning. When implemented on the Qwen3-4B-Instruct model,
our method can greatly enhance its performance on the HealthBench-Hard
benchmark from 7.0 to 27.2 using only 2k samples, thus achieving
state-of-the-art results for models of this scale. Our analysis confirms that
rubric-driven RL fos-ters consistent performance gains across diverse
consultation scenarios, going beyond simple numerical improvements. These
findings underscore rubric-based feedback as a scalable strategy for advancing
LLMs in intricate, open-ended tasks.
ORBIT, a rubric-based incremental training framework, enhances LLM performance in medical dialogue by using dynamic rubrics for RL, achieving state-of-the-art results on HealthBench-Hard.
reinforcement learning rubric-based incremental training synthetic dialogue generation rubric-guided feedback HealthBench-Hard Qwen3-4B-Instruct medical consultation open-ended tasks