Pathology-CoT: Learning Visual Chain-of-Thought Agent from Expert Whole
  Slide Image Diagnosis Behavior
Diagnosing a whole-slide image is an interactive, multi-stage process
involving changes in magnification and movement between fields. Although recent
pathology foundation models are strong, practical agentic systems that decide
what field to examine next, adjust magnification, and deliver explainable
diagnoses are still lacking. The blocker is data: scalable, clinically aligned
supervision of expert viewing behavior that is tacit and experience-based, not
written in textbooks or online, and therefore absent from large language model
training. We introduce the AI Session Recorder, which works with standard WSI
viewers to unobtrusively record routine navigation and convert the viewer logs
into standardized behavioral commands (inspect or peek at discrete
magnifications) and bounding boxes. A lightweight human-in-the-loop review
turns AI-drafted rationales into the Pathology-CoT dataset, a form of paired
"where to look" and "why it matters" supervision produced at roughly six times
lower labeling time. Using this behavioral data, we build Pathologist-o3, a
two-stage agent that first proposes regions of interest and then performs
behavior-guided reasoning. On gastrointestinal lymph-node metastasis detection,
it achieved 84.5% precision, 100.0% recall, and 75.4% accuracy, exceeding the
state-of-the-art OpenAI o3 model and generalizing across backbones. To our
knowledge, this constitutes one of the first behavior-grounded agentic systems
in pathology. Turning everyday viewer logs into scalable, expert-validated
supervision, our framework makes agentic pathology practical and establishes a
path to human-aligned, upgradeable clinical AI.
68edadefde1fee572713a739
Sheng Wang
68edadefde1fee572713a73a
Ruiming Wu
68edadefde1fee572713a73b
Charles Herndon
68edadefde1fee572713a73c
Yihang Liu
68edadefde1fee572713a73d
Shunsuke Koga
68edadefde1fee572713a73e
Jeanne Shen
68edadefde1fee572713a73f
Zhi Huang
Pathology-CoT: Learning Visual Chain-of-Thought Agent from Expert Whole
  Slide Image Diagnosis Behavior
Diagnosing a whole-slide image is an interactive, multi-stage process
involving changes in magnification and movement between fields. Although recent
pathology foundation models are strong, practical agentic systems that decide
what field to examine next, adjust magnification, and deliver explainable
diagnoses are still lacking. The blocker is data: scalable, clinically aligned
supervision of expert viewing behavior that is tacit and experience-based, not
written in textbooks or online, and therefore absent from large language model
training. We introduce the AI Session Recorder, which works with standard WSI
viewers to unobtrusively record routine navigation and convert the viewer logs
into standardized behavioral commands (inspect or peek at discrete
magnifications) and bounding boxes. A lightweight human-in-the-loop review
turns AI-drafted rationales into the Pathology-CoT dataset, a form of paired
"where to look" and "why it matters" supervision produced at roughly six times
lower labeling time. Using this behavioral data, we build Pathologist-o3, a
two-stage agent that first proposes regions of interest and then performs
behavior-guided reasoning. On gastrointestinal lymph-node metastasis detection,
it achieved 84.5% precision, 100.0% recall, and 75.4% accuracy, exceeding the
state-of-the-art OpenAI o3 model and generalizing across backbones. To our
knowledge, this constitutes one of the first behavior-grounded agentic systems
in pathology. Turning everyday viewer logs into scalable, expert-validated
supervision, our framework makes agentic pathology practical and establishes a
path to human-aligned, upgradeable clinical AI.
A framework records and utilizes expert navigation behavior in whole-slide imaging to build an agentic system for pathology diagnosis, achieving high precision and recall in metastasis detection.
whole-slide image AI Session Recorder viewer logs behavioral commands bounding boxes Pathology-CoT dataset Pathologist-o3 regions of interest behavior-guided reasoning gastrointestinal lymph-node metastasis detection OpenAI o3 model