Learning from Irregularly Sampled Data for Endomicroscopy Super-resolution: A Comparative Study of Sparse and Dense Approaches

Purpose: Probe-based Confocal Laser Endomicroscopy (pCLE) enables performing an optical biopsy, providing real-time microscopic images, via a probe. pCLE probes consist of multiple optical fibres arranged in a bundle, which taken together generate signals in an irregularly sampled pattern. Current pCLE reconstruction is based on interpolating irregular signals onto an over-sampled Cartesian grid, using a naive linear interpolation. It was shown that Convolutional Neural Networks (CNNs) could improve pCLE image quality. Although classical CNNs were applied to pCLE, input data were limited to reconstructed images in contrast to irregular data produced by pCLE. Methods: We compare pCLE reconstruction and super-resolution (SR) methods taking irregularly sampled or reconstructed pCLE images as input. We also propose to embed a Nadaraya-Watson (NW) kernel regression into the CNN framework as a novel trainable CNN layer. Using the NW layer and exemplar-based super-resolution, we design an NWNetSR architecture that allows for reconstructing high-quality pCLE images directly from the irregularly sampled input data. We created synthetic sparse pCLE images to evaluate our methodology. Results: The results were validated through an image quality assessment based on a combination of the following metrics: Peak signal-to-noise ratio, the Structural Similarity Index. Conclusion: Both dense and sparse CNNs outperform the reconstruction method currently used in the clinic. The main contributions of our study are a comparison of sparse and dense approach in pCLE image reconstruction, implementing trainable generalised NW kernel regression, and adaptation of synthetic data for training pCLE SR.

READ FULL TEXT

page 5

page 10

research
03/25/2021

Zero-shot super-resolution with a physically-motivated downsampling kernel for endomicroscopy

Super-resolution (SR) methods have seen significant advances thanks to t...
research
07/11/2019

Single Image Super-Resolution via CNN Architectures and TV-TV Minimization

Super-resolution (SR) is a technique that allows increasing the resoluti...
research
03/23/2018

Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction

Purpose: Probe-based Confocal Laser Endomicroscopy (pCLE) is a recent im...
research
10/12/2022

QMRNet: Quality Metric Regression for EO Image Quality Assessment and Super-Resolution

Latest advances in Super-Resolution (SR) have been tested with general p...
research
03/30/2020

Super Resolution for Root Imaging

High-resolution cameras have become very helpful for plant phenotyping b...
research
03/22/2005

A hybrid MLP-PNN architecture for fast image superresolution

Image superresolution methods process an input image sequence of a scene...
research
02/22/2022

Roto-Translation Equivariant Super-Resolution of Two-Dimensional Flows Using Convolutional Neural Networks

Convolutional neural networks (CNNs) often process vectors as quantities...

Please sign up or login with your details

Forgot password? Click here to reset