End-to-End Hyperspectral-Depth Imaging with Learned Diffractive Optics

by   Seung-Hwan Baek, et al.

To extend the capabilities of spectral imaging, hyperspectral and depth imaging have been combined to capture the higher-dimensional visual information. However, the form factor of the combined imaging systems increases, limiting the applicability of this new technology. In this work, we propose a monocular imaging system for simultaneously capturing hyperspectral-depth (HS-D) scene information with an optimized diffractive optical element (DOE). In the training phase, this DOE is optimized jointly with a convolutional neural network to estimate HS-D data from a snapshot input. To study natural image statistics of this high-dimensional visual data and to enable such a machine learning-based DOE training procedure, we record two HS-D datasets. One is used for end-to-end optimization in deep optical HS-D imaging, and the other is used for enhancing reconstruction performance with a real-DOE prototype. The optimized DOE is fabricated with a grayscale lithography process and inserted into a portable HS-D camera prototype, which is shown to robustly capture HS-D information. In extensive evaluations, we demonstrate that our deep optical imaging system achieves state-of-the-art results for HS-D imaging and that the optimized DOE outperforms alternative optical designs.



There are no comments yet.


page 1

page 4

page 6

page 7

page 9

page 10

page 11

page 13


Deep Optics for Single-shot High-dynamic-range Imaging

High-dynamic-range (HDR) imaging is crucial for many computer graphics a...

Learning Wavefront Coding for Extended Depth of Field Imaging

The depth of field constitutes an important quality factor of imaging sy...

Real-time Hyperspectral Imaging in Hardware via Trained Metasurface Encoders

Hyperspectral imaging has attracted significant attention to identify sp...

Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection

Depth reconstruction and hyperspectral reflectance reconstruction are tw...

Programmable Spectral Filter Arrays for Hyperspectral Imaging

Modulating the spectral dimension of light has numerous applications in ...

Combined Person Classification with Airborne Optical Sectioning

Fully autonomous drones have been demonstrated to find lost or injured p...

FPGA system for real-time computational extended depth of field imaging using phase aperture coding

We present a proof-of-concept end-to-end system for computational extend...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.