A deep learning framework for quality assessment and restoration in video endoscopy

04/15/2019
by   Sharib Ali, et al.
8

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5 of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7 videos.

READ FULL TEXT

page 4

page 7

page 8

page 10

page 11

research
05/08/2019

Endoscopy artifact detection (EAD 2019) challenge dataset

Endoscopic artifacts are a core challenge in facilitating the diagnosis ...
research
08/30/2018

Artifacts Detection and Error Block Analysis from Broadcasted Videos

With the advancement of IPTV and HDTV technology, previous subtle errors...
research
04/04/2023

Can Adversarial Networks Make Uninformative Colonoscopy Video Frames Clinically Informative?

Various artifacts, such as ghost colors, interlacing, and motion blur, h...
research
12/30/2022

Blind Restoration of Real-World Audio by 1D Operational GANs

Objective: Despite numerous studies proposed for audio restoration in th...
research
03/12/2022

Deep learning-based conditional inpainting for restoration of artifact-affected 4D CT images

4D CT imaging is an essential component of radiotherapy of thoracic/abdo...
research
10/26/2018

Deep Convolutional Neural Network Applied to Quality Assessment for Video Tracking

Surveillance videos often suffer from blur and exposure distortions that...
research
10/09/2017

UG^2: a Video Benchmark for Assessing the Impact of Image Restoration andEnhancement on Automatic Visual Recognition

Advances in image restoration and enhancement techniques have led to dis...

Please sign up or login with your details

Forgot password? Click here to reset