SRT-H: A Hierarchical Framework for Autonomous Surgery via Language
  Conditioned Imitation Learning
Research on autonomous surgery has largely focused on simple task automation
in controlled environments. However, real-world surgical applications demand
dexterous manipulation over extended durations and generalization to the
inherent variability of human tissue. These challenges remain difficult to
address using existing logic-based or conventional end-to-end learning
approaches. To address this gap, we propose a hierarchical framework for
performing dexterous, long-horizon surgical steps. Our approach utilizes a
high-level policy for task planning and a low-level policy for generating robot
trajectories. The high-level planner plans in language space, generating
task-level or corrective instructions that guide the robot through the
long-horizon steps and correct for the low-level policy's errors. We validate
our framework through ex vivo experiments on cholecystectomy, a
commonly-practiced minimally invasive procedure, and conduct ablation studies
to evaluate key components of the system. Our method achieves a 100\% success
rate across eight unseen ex vivo gallbladders, operating fully autonomously
without human intervention. This work demonstrates step-level autonomy in a
surgical procedure, marking a milestone toward clinical deployment of
autonomous surgical systems.
686fb55f6626f47a7a1f8259
Ji Woong Kim
686fb55f6626f47a7a1f825a
Juo-Tung Chen
686fb55f6626f47a7a1f825b
Pascal Hansen
686fb55f6626f47a7a1f825c
Lucy X. Shi
686fb55f6626f47a7a1f825d
Antony Goldenberg
686fb55f6626f47a7a1f825e
Samuel Schmidgall
686fb55f6626f47a7a1f825f
Paul Maria Scheikl
686fb55f6626f47a7a1f8260
Anton Deguet
686fb55f6626f47a7a1f8261
Brandon M. White
686fb55f6626f47a7a1f8262
De Ru Tsai
686fb55f6626f47a7a1f8263
Richard Cha
686fb55f6626f47a7a1f8264
Jeffrey Jopling
686fb55f6626f47a7a1f8265
Chelsea Finn
686fb55f6626f47a7a1f8266
Axel Krieger
SRT-H: A Hierarchical Framework for Autonomous Surgery via Language
  Conditioned Imitation Learning
Research on autonomous surgery has largely focused on simple task automation
in controlled environments. However, real-world surgical applications demand
dexterous manipulation over extended durations and generalization to the
inherent variability of human tissue. These challenges remain difficult to
address using existing logic-based or conventional end-to-end learning
approaches. To address this gap, we propose a hierarchical framework for
performing dexterous, long-horizon surgical steps. Our approach utilizes a
high-level policy for task planning and a low-level policy for generating robot
trajectories. The high-level planner plans in language space, generating
task-level or corrective instructions that guide the robot through the
long-horizon steps and correct for the low-level policy's errors. We validate
our framework through ex vivo experiments on cholecystectomy, a
commonly-practiced minimally invasive procedure, and conduct ablation studies
to evaluate key components of the system. Our method achieves a 100\% success
rate across eight unseen ex vivo gallbladders, operating fully autonomously
without human intervention. This work demonstrates step-level autonomy in a
surgical procedure, marking a milestone toward clinical deployment of
autonomous surgical systems.
A hierarchical framework with high-level language-based task planning and low-level trajectory generation achieves full autonomy in ex vivo cholecystectomy.
hierarchical framework high-level policy low-level policy language space task-level instructions corrective instructions robot trajectories ex vivo experiments cholecystectomy minimally invasive procedure step-level autonomy clinical deployment