Endo-Sim2Real: Consistency learning-based domain adaptation for instrument segmentation

07/22/2020
by   Manish Sahu, et al.
0

Surgical tool segmentation in endoscopic videos is an important component of computer assisted interventions systems. Recent success of image-based solutions using fully-supervised deep learning approaches can be attributed to the collection of big labeled datasets. However, the annotation of a big dataset of real videos can be prohibitively expensive and time consuming. Computer simulations could alleviate the manual labeling problem, however, models trained on simulated data do not generalize to real data. This work proposes a consistency-based framework for joint learning of simulated and real (unlabeled) endoscopic data to bridge this performance generalization issue. Empirical results on two data sets (15 videos of the Cholec80 and EndoVis'15 dataset) highlight the effectiveness of the proposed Endo-Sim2Real method for instrument segmentation. We compare the segmentation of the proposed approach with state-of-the-art solutions and show that our method improves segmentation both in terms of quality and quantity.

READ FULL TEXT

page 7

page 12

page 13

research
03/02/2021

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Purpose: Segmentation of surgical instruments in endoscopic videos is es...
research
06/23/2022

Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Need

Data diversity and volume are crucial to the success of training deep le...
research
07/17/2020

Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery

Semantic tool segmentation in surgical videos is important for surgical ...
research
02/23/2022

Augmentation based unsupervised domain adaptation

The insertion of deep learning in medical image analysis had lead to the...
research
03/24/2021

One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video

Surgical instrument segmentation in robot-assisted surgery (RAS) - espec...
research
06/28/2023

Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis

Despite their impressive performance in various surgical scene understan...
research
08/26/2020

Simulation-supervised deep learning for analysing organelles states and behaviour in living cells

In many real-world scientific problems, generating ground truth (GT) for...

Please sign up or login with your details

Forgot password? Click here to reset