Summary: A surgical robot trained on real procedure videos performed a critical phase of gallbladder removal autonomously, adapting to unexpected situations and responding to voice commands. This breakthrough shows how artificial intelligence can combine precision with the flexibility needed for real-world medicine.
The robot, using a machine-learning framework similar to ChatGPT, demonstrated expert-level performance even under challenging, variable conditions. Researchers see this as a key step toward fully autonomous, reliable surgical systems that can assist or operate independently.
Key Facts:
- Autonomous Adaptability: The robot adapted in real-time to anatomy differences, unexpected events, and verbal corrections.
- Imitation Learning: It learned by watching videos of human surgeons and was reinforced with task captions for context.
- Surgical Milestone: Successfully performed a complex, 17-task gallbladder removal phase with results comparable to experts.
Source: JHU
A robot trained on videos of surgeries performed a lengthy phase of a gallbladder removal without human help. The robot operated for the first time on a lifelike patient, and during the operation, responded to and learned from voice commands from the team—like a novice surgeon working with a mentor.
The robot performed unflappably across trials and with the expertise of a skilled human surgeon, even during unexpected scenarios typical in real life medical emergencies.
The federally-funded work, led by Johns Hopkins University researchers, is a transformative advancement in surgical robotics, where robots can perform with both mechanical precision and human-like adaptability and understanding.
“This advancement moves us from robots that can execute specific surgical tasks to robots that truly understand surgical procedures,” said medical roboticist Axel Krieger.
“This is a critical distinction that brings us significantly closer to clinically viable autonomous surgical systems that can work in the messy, unpredictable reality of actual patient care.”
The findings are published today in Science Robotics.
In 2022, Krieger’s Smart Tissue Autonomous Robot, STAR, performed the first autonomous robotic surgery on a live animal – a laparoscopic surgery on a pig. But that robot required specially marked tissue, operated in a highly controlled environment, and followed a rigid, predetermined surgical plan. Krieger said it was like teaching a robot to drive along a carefully mapped route.
But his new system, he says, “is like teaching a robot to navigate any road, in any condition, responding intelligently to whatever it encounters.”
Surgical Robot Transformer-Hierarchy, SRT-H, truly performs surgery, adapting to individual anatomical features in real-time, making decisions on the fly, and self-correcting when things don’t go as expected.
Built with the same machine learning architecture that powers ChatGPT, SRT-H is also interactive, able respond to spoken commands (“grab the gallbladder head”) and corrections (“move the left arm a bit to the left”). The robot learns from this feedback.
“This work represents a major leap from prior efforts because it tackles some of the fundamental barriers to deploying autonomous surgical robots in the real world,” said lead author Ji Woong “Brian” Kim, a former postdoctoral researcher at Johns Hopkins who’s now with Stanford University.
“Our work shows that AI models can be made reliable enough for surgical autonomy—something that once felt far-off but is now demonstrably viable.”
Last year Krieger’s team used the system to train a robot to perform three foundational surgical tasks: manipulating a needle, lifting body tissue, and suturing. Those tasks took just a few seconds each.
The gallbladder removal procedure is much more complex, a minutes-long string of 17 tasks. The robot had to identify certain ducts and arteries and grab them precisely, strategically place clips, and sever parts with scissors.
SRT-H learned how to do the gall bladder work by watching videos of Johns Hopkins surgeons doing it on pig cadavers. The team reinforced the visual training with captions describing the tasks. After watching the videos, the robot performed the surgery with 100% accuracy.
Although the robot took longer to perform the work than a human surgeon, the results were comparable to an expert surgeon.
“Just as surgical residents often master different parts of an operation at different rates, this work illustrates the promise of developing autonomous robotic systems in a similarly modular and progressive manner,” says Johns Hopkins surgeon Jeff Jopling, a co-author.
The robot performed flawlessly across anatomical conditions that weren’t uniform, and during unexpected detours—such as when the researchers changed the robot’s starting position and when they added blood-like dyes that changed the appearance of the gallbladder and surrounding tissues.
“To me it really shows that it’s possible to perform complex surgical procedures autonomously,” Krieger said.
“This is a proof of concept that it’s possible and this imitation learning framework can automate such complex procedure with such a high degree of robustness.”
Next the team would like to train and test the system on more types of surgeries and expand its capabilities to perform a complete autonomous surgery.
Authors include Johns Hopkins PhD student Juo-Tung Chen; Johns Hopkins visiting graduate student Pascal Hansen; Stanford University PhD student Lucy X. Shi; Johns Hopkins undergraduate Antony Goldenberg; Johns Hopkins PhD student Samuel Schmidgall; former Johns Hopkins postdoctoral fellow Paul Maria Scheikl; Johns Hopkins research engineer Anton Deguet; surgical fellow Brandon M. White, Stanford University assistant professor Chelsea Finn; and De Ru Tsai and Richard Cha of Optosurgical.
About this robotics and AI research news
Author: Jill Rosen
Source: JHU
Contact: Jill Rosen – JHU
Image: The image is credited to Neuroscience News
Original Research: Closed access.
“SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation Learning” by Axel Krieger et al. Science Robotics
Abstract
SRT-H: A Hierarchical Framework for Autonomous Surgery via Language-Conditioned Imitation Learning
Research on autonomous surgery has largely focused on simple task automation in controlled environments. However, real-world surgical applications demand dexterous manipulation over extended durations and robust generalization to the inherent variability of human tissue.
These challenges remain difficult to address using existing logic-based or conventional end-to-end learning strategies.
To address this gap, we propose a hierarchical framework for performing dexterous, long-horizon surgical steps. Our approach uses a high-level policy for task planning and a low-level policy for generating low-level trajectories.
The high-level planner plans in language space, generating task-level or corrective instructions that guide the robot through the long-horizon steps and help recover from errors made by the low-level policy.
We validated our framework through ex vivo experiments on cholecystectomy, a commonly practiced minimally invasive procedure, and conducted ablation studies to evaluate key components of the system.
Our method achieves a 100% success rate across eight different ex vivo gallbladders, operating fully autonomously without human intervention. The hierarchical approach improved the policy’s ability to recover from suboptimal states that are inevitable in the highly dynamic environment of realistic surgical applications.
This work demonstrates step-level autonomy in a surgical procedure, marking a milestone toward clinical deployment of autonomous surgical systems.
