Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopy

10/30/2017
by   Faisal, et al.
0

Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20 contrast of lesion topography. Imaging tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make geometric assumptions or incorporate a priori information, which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 100,000 images of an anatomically-realistic colon. We validate our approach with real endoscopy images from a porcine colon, transferred to a synthetic-like domain, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 7

page 8

page 9

research
03/21/2016

Unified Depth Prediction and Intrinsic Image Decomposition from a Single Image via Joint Convolutional Neural Fields

We present a method for jointly predicting a depth map and intrinsic ima...
research
11/24/2014

Deep Convolutional Neural Fields for Depth Estimation from a Single Image

We consider the problem of depth estimation from a single monocular imag...
research
06/18/2019

A Conditional Random Field Model for Context Aware Cloud Detection in Sky Images

A conditional random field (CRF) model for cloud detection in ground bas...
research
01/13/2018

Size-to-depth: A New Perspective for Single Image Depth Estimation

In this paper we consider the problem of single monocular image depth es...
research
06/19/2018

Cancer Metastasis Detection With Neural Conditional Random Field

Breast cancer diagnosis often requires accurate detection of metastasis ...
research
01/11/2021

Colorectal Polyp Detection in Real-world Scenario: Design and Experiment Study

Colorectal polyps are abnormal tissues growing on the intima of the colo...
research
04/07/2017

High-Quality Correspondence and Segmentation Estimation for Dual-Lens Smart-Phone Portraits

Estimating correspondence between two images and extracting the foregrou...

Please sign up or login with your details

Forgot password? Click here to reset