Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging

06/19/2017

∙

Intra-operative measurements of tissue shape and multi/ hyperspectral information have the potential to provide surgical guidance and decision making support. We report an optical probe based system to combine sparse hyperspectral measurements and spectrally-encoded structured lighting (SL) for surface measurements. The system provides informative signals for navigation with a surgical interface. By rapidly switching between SL and white light (WL) modes, SL information is combined with structure-from-motion (SfM) from white light images, based on SURF feature detection and Lucas-Kanade (LK) optical flow to provide quasi-dense surface shape reconstruction with known scale in real-time. Furthermore, "super-spectral-resolution" was realized, whereby the RGB images and sparse hyperspectral data were integrated to recover dense pixel-level hyperspectral stacks, by using convolutional neural networks to upscale the wavelength dimension. Validation and demonstration of this system is reported on ex vivo/in vivo animal/ human experiments.

READ FULL TEXT

Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging

Probe-based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks

Recovering Dense Tissue Multispectral Signal from in vivo RGB Images

Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction

Estimation of Tissue Oxygen Saturation from RGB images and Sparse Hyperspectral Signals based on Conditional Generative Adversarial Network

Intra-operative Brain Tumor Detection with Deep Learning-Optimized Hyperspectral Imaging

KRISM --- Krylov Subspace-based Optical Computing of Hyperspectral Images

Programmable Spectral Filter Arrays for Hyperspectral Imaging

Endoscopic Depth Measurement and Super-Spectral-Resolution Imaging

Related Research

Probe-based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks

Recovering Dense Tissue Multispectral Signal from in vivo RGB Images

Deep Learning Approach for Hyperspectral Image Demosaicking, Spectral Correction and High-resolution RGB Reconstruction

Estimation of Tissue Oxygen Saturation from RGB images and Sparse Hyperspectral Signals based on Conditional Generative Adversarial Network

Intra-operative Brain Tumor Detection with Deep Learning-Optimized Hyperspectral Imaging

KRISM --- Krylov Subspace-based Optical Computing of Hyperspectral Images

Programmable Spectral Filter Arrays for Hyperspectral Imaging