Gazal-R1: Achieving State-of-the-Art Medical Reasoning with
  Parameter-Efficient Two-Stage Training
We present Gazal-R1, a 32-billion-parameter language model that achieves
state-of-the-art performance in medical reasoning while providing transparent,
step-by-step explanations for clinical decision-making. Built upon Qwen3 32B,
our model demonstrates that strategic training can enable mid-sized models to
outperform significantly larger counterparts in specialized domains. We
developed a novel two-stage training pipeline: first, supervised fine-tuning on
a carefully curated dataset of 107,033 synthetic medical reasoning examples
that teaches structured clinical thinking, enhanced by advanced
parameter-efficient techniques including Weight-Decomposed Low-Rank Adaptation
(DoRA) and Rank-Stabilized LoRA (rsLoRA); second, reinforcement learning using
Group Relative Policy Optimization (GRPO) with a sophisticated multi-component
reward system that refines accuracy, format adherence, and reasoning quality.
Gazal-R1 achieves exceptional performance across medical benchmarks, scoring
87.1% on MedQA, 81.6% on MMLU Pro (Medical), and 79.6% on PubMedQA, surpassing
models up to 12x larger. Beyond its strong empirical results, this work
provides detailed insights into the challenges of training reasoning-capable
models in specialized domains, including issues with reward hacking, training
instability, and the fundamental tension between factual recall and detailed
reasoning. Our methodology offers a reproducible framework for developing
high-capability, domain-specific language models that balance performance,
efficiency, and explainability.
68625a4c9e7509383d29ac4c
Ahmed M. Adly
claimed_verified
2025-07-02T12:22:43.747Z
63aca106e3b217fb36cf1950
/avatars/b37cc9102f875b6ce0c55a294c052078.svg
Ahmed Mostafa
user
AhmedMostafa
68625a4c9e7509383d29ac4d
Mostafa Samy
claimed_verified
2025-07-02T08:32:47.090Z
65302189d849382d55569306
https://cdn-avatars.huggingface.co/v1/production/uploads/65302189d849382d55569306/dMvY2rNcMEySqfEia-GdO.jpeg
Samy
user
DrMostafa
68625a4c9e7509383d29ac4e
Amr Fawzy
Gazal-R1: Achieving State-of-the-Art Medical Reasoning with
  Parameter-Efficient Two-Stage Training
We present Gazal-R1, a 32-billion-parameter language model that achieves
state-of-the-art performance in medical reasoning while providing transparent,
step-by-step explanations for clinical decision-making. Built upon Qwen3 32B,
our model demonstrates that strategic training can enable mid-sized models to
outperform significantly larger counterparts in specialized domains. We
developed a novel two-stage training pipeline: first, supervised fine-tuning on
a carefully curated dataset of 107,033 synthetic medical reasoning examples
that teaches structured clinical thinking, enhanced by advanced
parameter-efficient techniques including Weight-Decomposed Low-Rank Adaptation
(DoRA) and Rank-Stabilized LoRA (rsLoRA); second, reinforcement learning using
Group Relative Policy Optimization (GRPO) with a sophisticated multi-component
reward system that refines accuracy, format adherence, and reasoning quality.
Gazal-R1 achieves exceptional performance across medical benchmarks, scoring
87.1% on MedQA, 81.6% on MMLU Pro (Medical), and 79.6% on PubMedQA, surpassing
models up to 12x larger. Beyond its strong empirical results, this work
provides detailed insights into the challenges of training reasoning-capable
models in specialized domains, including issues with reward hacking, training
instability, and the fundamental tension between factual recall and detailed
reasoning. Our methodology offers a reproducible framework for developing
high-capability, domain-specific language models that balance performance,
efficiency, and explainability.
Gazal-R1, a 32-billion-parameter language model, achieves top performance in medical reasoning through strategic training, including advanced parameter-efficient techniques and reinforcement learning, providing detailed explanations for clinical decisions.
Weight-Decomposed Low-Rank Adaptation (DoRA) Rank-Stabilized LoRA (rsLoRA) Group Relative Policy Optimization (GRPO) MedQA MMLU Pro (Medical) PubMedQA reasoning-capable models reward hacking training instability