A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images

02/28/2021
by   Luca Sestini, et al.
0

3-D pose estimation of instruments is a crucial step towards automatic scene understanding in robotic minimally invasive surgery. Although robotic systems can potentially directly provide joint values, this information is not commonly exploited inside the operating room, due to its possible unreliability, limited access and the time-consuming calibration required, especially for continuum robots. For this reason, standard approaches for 3-D pose estimation involve the use of external tracking systems. Recently, image-based methods have emerged as promising, non-invasive alternatives. While many image-based approaches in the literature have shown accurate results, they generally require either a complex iterative optimization for each processed image, making them unsuitable for real-time applications, or a large number of manually-annotated images for efficient learning. In this paper we propose a self-supervised image-based method, exploiting, at training time only, the imprecise kinematic information provided by the robot. In order to avoid introducing time-consuming manual annotations, the problem is formulated as an auto-encoder, smartly bottlenecked by the presence of a physical model of the robotic instruments and surgical camera, forcing a separation between image background and kinematic content. Validation of the method was performed on semi-synthetic, phantom and in-vivo datasets, obtained using a flexible robotized endoscope, showing promising results for real-time image-based 3-D pose estimation of surgical instruments.

READ FULL TEXT

page 1

page 7

research
02/13/2019

Self-Supervised Surgical Tool Segmentation using Kinematic Information

Surgical tool segmentation in endoscopic images is the first step toward...
research
03/03/2020

Single-Shot Pose Estimation of Surgical Robot Instruments' Shafts from Monocular Endoscopic Images

Surgical robots are used to perform minimally invasive surgery and allev...
research
03/15/2021

MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

Surgical robots are controlled using a priori models based on robots' ge...
research
03/24/2017

Deep Residual Learning for Instrument Segmentation in Robotic Surgery

Detection, tracking, and pose estimation of surgical instruments are cru...
research
07/08/2021

Kinematic Parameter Optimization of a Miniaturized Surgical Instrument Based on Dexterous Workspace Determination

Miniaturized instruments are highly needed for robot assisted medical he...
research
07/09/2020

Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks

Surgical tool segmentation in endoscopic images is an important problem:...
research
05/27/2023

An Image Based Visual Servo Method for Probe-and-Drogue Autonomous Aerial Refueling

With the high focus on autonomous aerial refueling recently, it becomes ...

Please sign up or login with your details

Forgot password? Click here to reset