One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video

03/24/2021
by   Zixu Zhao, et al.
1

Surgical instrument segmentation in robot-assisted surgery (RAS) - especially that using learning-based models - relies on the assumption that training and testing videos are sampled from the same domain. However, it is impractical and expensive to collect and annotate sufficient data from every new domain. To greatly increase the label efficiency, we explore a new problem, i.e., adaptive instrument segmentation, which is to effectively adapt one source model to new robotic surgical videos from multiple target domains, only given the annotated instruments in the first frame. We propose MDAL, a meta-learning based dynamic online adaptive learning scheme with a two-stage framework to fast adapt the model parameters on the first frame and partial subsequent frames while predicting the results. MDAL learns the general knowledge of instruments and the fast adaptation ability through the video-specific meta-learning paradigm. The added gradient gate excludes the noisy supervision from pseudo masks for dynamic online adaptation on target videos. We demonstrate empirically that MDAL outperforms other state-of-the-art methods on two datasets (including a real-world RAS dataset). The promising performance on ex-vivo scenes also benefits the downstream tasks such as robot-assisted suturing and camera control.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 5

page 6

02/24/2019

U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument

Conventional therapy approaches limit surgeons' dexterity control due to...
07/10/2020

ISINet: An Instance-Based Approach for Surgical Instrument Segmentation

We study the task of semantic segmentation of surgical instruments in ro...
03/16/2021

Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data

Surgical instrument segmentation for robot-assisted surgery is needed fo...
11/17/2020

Multi-frame Feature Aggregation for Real-time Instrument Segmentation in Endoscopic Video

Deep learning-based methods have achieved promising results on surgical ...
03/02/2021

Simulation-to-Real domain adaptation with teacher-student learning for endoscopic instrument segmentation

Purpose: Segmentation of surgical instruments in endoscopic videos is es...
04/02/2020

Scene-Adaptive Video Frame Interpolation via Meta-Learning

Video frame interpolation is a challenging problem because there are dif...
07/22/2020

Endo-Sim2Real: Consistency learning-based domain adaptation for instrument segmentation

Surgical tool segmentation in endoscopic videos is an important componen...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.