SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics

by   Yang Li, et al.

Traditional control and task automation have been successfully demonstrated in a variety of structured, controlled environments through the use of highly specialized modeled robotic systems in conjunction with multiple sensors. However, application of autonomy in endoscopic surgery is very challenging, particularly in soft tissue work, due to the lack of high-quality images and the unpredictable, constantly deforming environment. In this work, we propose a novel surgical perception framework, SuPer, for surgical robotic control. This framework continuously collects 3D geometric information that allows for mapping of a deformable surgical field while tracking rigid instruments within the field. To achieve this, a model-based tracker is employed to localize the surgical tool with a kinematic prior in conjunction with a model-free tracker to reconstruct the deformable environment and provide an estimated point cloud as a mapping of the environment. The proposed framework was implemented on the da Vinci Surgical System in real-time with an end-effector controller where the target configurations are set and regulated through the framework. Our proposed framework successfully completed autonomous soft tissue manipulation tasks with high accuracy. The demonstration of this novel framework is promising for the future of surgical autonomy. In addition, we provide our dataset for further surgical research.


page 1

page 3

page 4

page 7


A 2D Surgical Simulation Framework for Tool-Tissue Interaction

The control and task automation of robotic surgical system is very chall...

Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Classification, Reconstruction, and Tracking

Accurate and robust tracking and reconstruction of the surgical scene is...

Autonomous Tissue Manipulation via Surgical Robot Using Learning Based Model Predictive Control

Tissue manipulation is a frequently used fundamental subtask of any surg...

Autonomous tissue retraction with a biomechanically informed logic based framework

Autonomy in parts of robot-assisted surgery is essential to reduce surge...

Integrated Planning and Control of Robotic Surgical Instruments for Tasks Autonomy

Agile maneuvers are essential for robot-enabled complex tasks such as su...

Deliberation in autonomous robotic surgery: a framework for handling anatomical uncertainty

Autonomous robotic surgery requires deliberation, i.e. the ability to pl...

Real-time Surgical Environment Enhancement for Robot-Assisted Minimally Invasive Surgery Based on Super-Resolution

In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant...

Please sign up or login with your details

Forgot password? Click here to reset