Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with
  Minimalist Rule-Based RL
Improving performance on complex tasks and enabling interpretable decision
making in large language models (LLMs), especially for clinical applications,
requires effective reasoning. Yet this remains challenging without supervised
fine-tuning (SFT) on costly chain-of-thought (CoT) data distilled from
closed-source models (e.g., GPT-4o). In this work, we present AlphaMed, the
first medical LLM to show that reasoning capability can emerge purely through
reinforcement learning (RL), using minimalist rule-based rewards on public
multiple-choice QA datasets, without relying on SFT or distilled CoT data.
AlphaMed achieves state-of-the-art results on six medical QA benchmarks,
outperforming models trained with conventional SFT+RL pipelines. On challenging
benchmarks (e.g., MedXpert), AlphaMed even surpasses larger or closed-source
models such as DeepSeek-V3-671B and Claude-3.5-Sonnet. To understand the
factors behind this success, we conduct a comprehensive data-centric analysis
guided by three questions: (i) Can minimalist rule-based RL incentivize
reasoning without distilled CoT supervision? (ii) How do dataset quantity and
diversity impact reasoning? (iii) How does question difficulty shape the
emergence and generalization of reasoning? Our findings show that dataset
informativeness is a key driver of reasoning performance, and that minimalist
RL on informative, multiple-choice QA data is effective at inducing reasoning
without CoT supervision. We also observe divergent trends across benchmarks,
underscoring limitations in current evaluation and the need for more
challenging, reasoning-oriented medical QA benchmarks.
6836e9b4481e0bfedacc61bc
Che Liu
admin_assigned
2025-05-28T12:37:55.201Z
631b9ff5824f2502e3557c7e
/avatars/076043c9dba07644a570692563ef8114.svg
liu
user
che111
6836e9b4481e0bfedacc61bd
Haozhe Wang
claimed_verified
2025-06-09T10:13:01.222Z
65bf52f0259bc6caeb74f8bf
/avatars/b38392e954466df784a5760ded5df804.svg
Haozhe Wang
user
JasperHaozhe
6836e9b4481e0bfedacc61be
Jiazhen Pan
claimed_verified
2025-05-28T19:14:16.399Z
66588c8a338165aad1516756
/avatars/c6539b4ef65f465f6f762628d6921be6.svg
JZPeterPan
user
JZPeterPan
6836e9b4481e0bfedacc61bf
Zhongwei Wan
6836e9b4481e0bfedacc61c0
Yong Dai
6836e9b4481e0bfedacc61c1
Fangzhen Lin
6836e9b4481e0bfedacc61c2
Wenjia Bai
admin_assigned
2025-05-28T12:48:49.737Z
67c973029d03a429bb2bbf99
/avatars/d05246e3d1e54e13acd1959155931dac.svg
Bai
user
baiwenjia
6836e9b4481e0bfedacc61c3
Daniel Rueckert
6836e9b4481e0bfedacc61c4
Rossella Arcucci
Beyond Distillation: Pushing the Limits of Medical LLM Reasoning with
  Minimalist Rule-Based RL
Improving performance on complex tasks and enabling interpretable decision
making in large language models (LLMs), especially for clinical applications,
requires effective reasoning. Yet this remains challenging without supervised
fine-tuning (SFT) on costly chain-of-thought (CoT) data distilled from
closed-source models (e.g., GPT-4o). In this work, we present AlphaMed, the
first medical LLM to show that reasoning capability can emerge purely through
reinforcement learning (RL), using minimalist rule-based rewards on public
multiple-choice QA datasets, without relying on SFT or distilled CoT data.
AlphaMed achieves state-of-the-art results on six medical QA benchmarks,
outperforming models trained with conventional SFT+RL pipelines. On challenging
benchmarks (e.g., MedXpert), AlphaMed even surpasses larger or closed-source
models such as DeepSeek-V3-671B and Claude-3.5-Sonnet. To understand the
factors behind this success, we conduct a comprehensive data-centric analysis
guided by three questions: (i) Can minimalist rule-based RL incentivize
reasoning without distilled CoT supervision? (ii) How do dataset quantity and
diversity impact reasoning? (iii) How does question difficulty shape the
emergence and generalization of reasoning? Our findings show that dataset
informativeness is a key driver of reasoning performance, and that minimalist
RL on informative, multiple-choice QA data is effective at inducing reasoning
without CoT supervision. We also observe divergent trends across benchmarks,
underscoring limitations in current evaluation and the need for more
challenging, reasoning-oriented medical QA benchmarks.
AlphaMed, a medical LLM, demonstrates superior reasoning capabilities through reinforcement learning using minimalist rule-based rewards, surpassing conventionally trained models on medical QA benchmarks.
reinforcement learning rule-based rewards medical LLM chain-of-thought reasoning capability multiple-choice QA datasets dataset informativeness medical QA benchmarks