FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation

02/16/2022
by   Luca Sestini, et al.
3

Automatic surgical instrument segmentation of endoscopic images is a crucial building block of many computer-assistance applications for minimally invasive surgery. So far, state-of-the-art approaches completely rely on the availability of a ground-truth supervision signal, obtained via manual annotation, thus expensive to collect at large scale. In this paper, we present FUN-SIS, a Fully-UNsupervised approach for binary Surgical Instrument Segmentation. FUN-SIS trains a per-frame segmentation model on completely unlabelled endoscopic videos, by solely relying on implicit motion information and instrument shape-priors. We define shape-priors as realistic segmentation masks of the instruments, not necessarily coming from the same dataset/domain as the videos. The shape-priors can be collected in various and convenient ways, such as recycling existing annotations from other datasets. We leverage them as part of a novel generative-adversarial approach, allowing to perform unsupervised instrument segmentation of optical-flow images during training. We then use the obtained instrument masks as pseudo-labels in order to train a per-frame segmentation model; to this aim, we develop a learning-from-noisy-labels architecture, designed to extract a clean supervision signal from these pseudo-labels, leveraging their peculiar noise properties. We validate the proposed contributions on three surgical datasets, including the MICCAI 2017 EndoVis Robotic Instrument Segmentation Challenge dataset. The obtained fully-unsupervised results for surgical instrument segmentation are almost on par with the ones of fully-supervised state-of-the-art approaches. This suggests the tremendous potential of the proposed method to leverage the great amount of unlabelled data produced in the context of minimally invasive surgery.

READ FULL TEXT

page 2

page 5

page 9

page 10

page 12

page 19

page 23

page 24

research
08/27/2020

Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion

Surgical instrument segmentation is a key component in developing contex...
research
08/09/2023

SegMatch: A semi-supervised learning method for surgical instrument segmentation

Surgical instrument segmentation is recognised as a key enabler to provi...
research
07/01/2020

Rethinking Anticipation Tasks: Uncertainty-aware Anticipation of Sparse Surgical Instrument Usage for Context-aware Assistance

Intra-operative anticipation of instrument usage is a necessary componen...
research
03/24/2021

One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video

Surgical instrument segmentation in robot-assisted surgery (RAS) - espec...
research
09/04/2023

SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of Surgical Tools

Instance segmentation of surgical instruments is a long-standing researc...
research
07/09/2020

Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks

Surgical tool segmentation in endoscopic images is an important problem:...
research
03/02/2021

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Purpose: Segmentation of surgical instruments in endoscopic videos is es...

Please sign up or login with your details

Forgot password? Click here to reset