End-to-End Hyperspectral-Depth Imaging with Learned Diffractive Optics

09/01/2020
by   Seung-Hwan Baek, et al.
32

To extend the capabilities of spectral imaging, hyperspectral and depth imaging have been combined to capture the higher-dimensional visual information. However, the form factor of the combined imaging systems increases, limiting the applicability of this new technology. In this work, we propose a monocular imaging system for simultaneously capturing hyperspectral-depth (HS-D) scene information with an optimized diffractive optical element (DOE). In the training phase, this DOE is optimized jointly with a convolutional neural network to estimate HS-D data from a snapshot input. To study natural image statistics of this high-dimensional visual data and to enable such a machine learning-based DOE training procedure, we record two HS-D datasets. One is used for end-to-end optimization in deep optical HS-D imaging, and the other is used for enhancing reconstruction performance with a real-DOE prototype. The optimized DOE is fabricated with a grayscale lithography process and inserted into a portable HS-D camera prototype, which is shown to robustly capture HS-D information. In extensive evaluations, we demonstrate that our deep optical imaging system achieves state-of-the-art results for HS-D imaging and that the optimized DOE outperforms alternative optical designs.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 4

page 6

page 7

page 9

page 10

page 11

page 13

08/01/2019

Deep Optics for Single-shot High-dynamic-range Imaging

High-dynamic-range (HDR) imaging is crucial for many computer graphics a...
12/31/2019

Learning Wavefront Coding for Extended Depth of Field Imaging

The depth of field constitutes an important quality factor of imaging sy...
04/05/2022

Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders

Hyperspectral imaging has attracted significant attention to identify sp...
04/08/2022

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Depth reconstruction and hyperspectral reflectance reconstruction are tw...
09/29/2021

Programmable Spectral Filter Arrays for Hyperspectral Imaging

Modulating the spectral dimension of light has numerous applications in ...
06/18/2021

Combined Person Classification with Airborne Optical Sectioning

Fully autonomous drones have been demonstrated to find lost or injured p...
08/03/2016

FPGA system for real-time computational extended depth of field imaging using phase aperture coding

We present a proof-of-concept end-to-end system for computational extend...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.