Colonoscopy Polyp Detection: Domain Adaptation From Medical Report Images to Real-time Videos

by   Zhi-Qin Zhan, et al.

Automatic colorectal polyp detection in colonoscopy video is a fundamental task, which has received a lot of attention. Manually annotating polyp region in a large scale video dataset is time-consuming and expensive, which limits the development of deep learning techniques. A compromise is to train the target model by using labeled images and infer on colonoscopy videos. However, there are several issues between the image-based training and video-based inference, including domain differences, lack of positive samples, and temporal smoothness. To address these issues, we propose an Image-video-joint polyp detection network (Ivy-Net) to address the domain gap between colonoscopy images from historical medical reports and real-time videos. In our Ivy-Net, a modified mixup is utilized to generate training data by combining the positive images and negative video frames at the pixel level, which could learn the domain adaptive representations and augment the positive samples. Simultaneously, a temporal coherence regularization (TCR) is proposed to introduce the smooth constraint on feature-level in adjacent frames and improve polyp detection by unlabeled colonoscopy videos. For evaluation, a new large colonoscopy polyp dataset is collected, which contains 3056 images from historical medical reports of 889 positive patients and 7.5-hour videos of 69 patients (28 positive). The experiments on the collected dataset demonstrate that our Ivy-Net achieves the state-of-the-art result on colonoscopy video.



There are no comments yet.


page 6


DRIV100: In-The-Wild Multi-Domain Dataset and Evaluation for Real-World Domain Adaptation of Semantic Segmentation

Together with the recent advances in semantic segmentation, many domain ...

Unsupervised Domain Adaptation for Face Recognition in Unlabeled Videos

Despite rapid advances in face recognition, there remains a clear gap be...

Triple-cooperative Video Shadow Detection

Shadow detection in a single image has received significant research int...

Exploiting Images for Video Recognition with Hierarchical Generative Adversarial Networks

Existing deep learning methods of video recognition usually require a la...

Diverse Large-Scale ITS Dataset Created from Continuous Learning for Real-Time Vehicle Detection

In traffic engineering, vehicle detectors are trained on limited dataset...

BSUV-Net 2.0: Spatio-Temporal Data Augmentations for Video-AgnosticSupervised Background Subtraction

Background subtraction (BGS) is a fundamental video processing task whic...

Meta-SVDD: Probabilistic Meta-Learning for One-Class Classification in Cancer Histology Images

To train a robust deep learning model, one usually needs a balanced set ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.