End-to-End Agentic RAG System Training for Traceable Diagnostic
  Reasoning
Accurate diagnosis with medical large language models is hindered by
knowledge gaps and hallucinations. Retrieval and tool-augmented methods help,
but their impact is limited by weak use of external knowledge and poor
feedback-reasoning traceability. To address these challenges, We introduce
Deep-DxSearch, an agentic RAG system trained end-to-end with reinforcement
learning (RL) that enables steer tracebale retrieval-augmented reasoning for
medical diagnosis. In Deep-DxSearch, we first construct a large-scale medical
retrieval corpus comprising patient records and reliable medical knowledge
sources to support retrieval-aware reasoning across diagnostic scenarios. More
crutially, we frame the LLM as the core agent and the retrieval corpus as its
environment, using tailored rewards on format, retrieval, reasoning structure,
and diagnostic accuracy, thereby evolving the agentic RAG policy from
large-scale data through RL.
  Experiments demonstrate that our end-to-end agentic RL training framework
consistently outperforms prompt-engineering and training-free RAG approaches
across multiple data centers. After training, Deep-DxSearch achieves
substantial gains in diagnostic accuracy, surpassing strong diagnostic
baselines such as GPT-4o, DeepSeek-R1, and other medical-specific frameworks
for both common and rare disease diagnosis under in-distribution and
out-of-distribution settings. Moreover, ablation studies on reward design and
retrieval corpus components confirm their critical roles, underscoring the
uniqueness and effectiveness of our approach compared with traditional
implementations. Finally, case studies and interpretability analyses highlight
improvements in Deep-DxSearch's diagnostic policy, providing deeper insight
into its performance gains and supporting clinicians in delivering more
reliable and precise preliminary diagnoses. See
https://github.com/MAGIC-AI4Med/Deep-DxSearch.
68abd6e286b21a0e2e358ac4
Qiaoyu Zheng
68abd6e286b21a0e2e358ac5
Yuze Sun
68abd6e286b21a0e2e358ac6
Chaoyi Wu
claimed_verified
2025-08-27T13:24:57.658Z
6436aaaa0c77d7c5036abdbd
https://cdn-avatars.huggingface.co/v1/production/uploads/6436aaaa0c77d7c5036abdbd/C9A276yEeAPkKLEqUcjl_.jpeg
Chaoyi Wu
user
chaoyi-wu
68abd6e286b21a0e2e358ac7
Weike Zhao
claimed_verified
2025-08-25T08:18:51.751Z
64365addfae287005149dd24
/avatars/bd6d4512d66fd9fd7fd5476ea7a44b46.svg
Weike Zhao
user
Angelakeke
68abd6e286b21a0e2e358ac8
Pengcheng Qiu
68abd6e286b21a0e2e358ac9
Yongguo Yu
68abd6e286b21a0e2e358aca
Kun Sun
68abd6e286b21a0e2e358acb
Yanfeng Wang
68abd6e286b21a0e2e358acc
Ya Zhang
68abd6e286b21a0e2e358acd
Weidi Xie
End-to-End Agentic RAG System Training for Traceable Diagnostic
  Reasoning
Accurate diagnosis with medical large language models is hindered by
knowledge gaps and hallucinations. Retrieval and tool-augmented methods help,
but their impact is limited by weak use of external knowledge and poor
feedback-reasoning traceability. To address these challenges, We introduce
Deep-DxSearch, an agentic RAG system trained end-to-end with reinforcement
learning (RL) that enables steer tracebale retrieval-augmented reasoning for
medical diagnosis. In Deep-DxSearch, we first construct a large-scale medical
retrieval corpus comprising patient records and reliable medical knowledge
sources to support retrieval-aware reasoning across diagnostic scenarios. More
crutially, we frame the LLM as the core agent and the retrieval corpus as its
environment, using tailored rewards on format, retrieval, reasoning structure,
and diagnostic accuracy, thereby evolving the agentic RAG policy from
large-scale data through RL.
  Experiments demonstrate that our end-to-end agentic RL training framework
consistently outperforms prompt-engineering and training-free RAG approaches
across multiple data centers. After training, Deep-DxSearch achieves
substantial gains in diagnostic accuracy, surpassing strong diagnostic
baselines such as GPT-4o, DeepSeek-R1, and other medical-specific frameworks
for both common and rare disease diagnosis under in-distribution and
out-of-distribution settings. Moreover, ablation studies on reward design and
retrieval corpus components confirm their critical roles, underscoring the
uniqueness and effectiveness of our approach compared with traditional
implementations. Finally, case studies and interpretability analyses highlight
improvements in Deep-DxSearch's diagnostic policy, providing deeper insight
into its performance gains and supporting clinicians in delivering more
reliable and precise preliminary diagnoses. See
https://github.com/MAGIC-AI4Med/Deep-DxSearch.
Deep-DxSearch, an agentic RAG system trained with reinforcement learning, enhances medical diagnosis accuracy by integrating a large-scale retrieval corpus and tailored rewards.
Deep-DxSearch RAG system reinforcement learning retrieval corpus retrieval-aware reasoning diagnostic accuracy prompt-engineering training-free RAG GPT-4o DeepSeek-R1 in-distribution out-of-distribution reward design interpretability analyses