FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements

10/29/2020
by   Salman S. Khan, et al.
13

Lensless imaging has emerged as a potential solution towards realizing ultra-miniature cameras by eschewing the bulky lens in a traditional camera. Without a focusing lens, the lensless cameras rely on computational algorithms to recover the scenes from multiplexed measurements. However, the current iterative-optimization-based reconstruction algorithms produce noisier and perceptually poorer images. In this work, we propose a non-iterative deep learning based reconstruction approach that results in orders of magnitude improvement in image quality for lensless reconstructions. Our approach, called FlatNet, lays down a framework for reconstructing high-quality photorealistic images from mask-based lensless cameras, where the camera's forward model formulation is known. FlatNet consists of two stages: (1) an inversion stage that maps the measurement into a space of intermediate reconstruction by learning parameters within the forward model formulation, and (2) a perceptual enhancement stage that improves the perceptual quality of this intermediate reconstruction. These stages are trained together in an end-to-end manner. We show high-quality reconstructions by performing extensive experiments on real and challenging scenes using two different types of lensless prototypes: one which uses a separable forward model and another, which uses a more general non-separable cropped-convolution model. Our end-to-end approach is fast, produces photorealistic reconstructions, and is easy to adopt for other mask-based lensless cameras.

READ FULL TEXT

page 7

page 8

page 9

page 12

page 13

page 17

page 20

page 21

research
08/30/2019

Learned reconstructions for practical mask-based lensless imaging

Mask-based lensless imagers are smaller and lighter than traditional len...
research
03/08/2022

Unrolled Primal-Dual Networks for Lensless Cameras

Conventional image reconstruction models for lensless cameras often assu...
research
08/07/2018

Fast and Accurate Camera Covariance Computation for Large 3D Reconstruction

Estimating uncertainty of camera parameters computed in Structure from M...
research
04/07/2021

Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution

High quality imaging usually requires bulky and expensive lenses to comp...
research
03/08/2022

PyNET-QxQ: A Distilled PyNET for QxQ Bayer Pattern Demosaicing in CMOS Image Sensor

The deep learning-based ISP models for mobile cameras produce high-quali...
research
01/26/2016

ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements

The goal of this paper is to present a non-iterative and more importantl...
research
01/27/2018

Meshed Up: Learnt Error Correction in 3D Reconstructions

Dense reconstructions often contain errors that prior work has so far mi...

Please sign up or login with your details

Forgot password? Click here to reset