Light Field Reconstruction via Attention-Guided Deep Fusion of Hybrid Lenses

02/14/2021
by   Jing Jin, et al.
3

This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. The performance of existing methods is still limited, as they produce either blurry results on plain textured areas or distortions around depth discontinuous boundaries. To tackle this challenge, we propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation, while the other module warps another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations adaptively via the learned attention maps, leading to the final high-resolution LF image with satisfactory results on both plain textured areas and depth discontinuous boundaries. Besides, to promote the effectiveness of our method trained with simulated hybrid data on real hybrid data captured by a hybrid LF imaging system, we carefully design the network architecture and the training strategy. Extensive experiments on both real and simulated hybrid data demonstrate the significant superiority of our approach over state-of-the-art ones. To the best of our knowledge, this is the first end-to-end deep learning method for LF reconstruction from a real hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission.

READ FULL TEXT

page 2

page 3

page 8

page 9

page 10

page 11

page 13

page 14

research
07/23/2019

Learning High-fidelity Light Field Images From Hybrid Inputs

This paper explores the reconstruction of high-fidelity LF images (i.e.,...
research
02/26/2020

Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

The acquisition of light field images with high angular resolution is co...
research
06/01/2023

Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution

Color-guided depth super-resolution (DSR) is an encouraging paradigm tha...
research
08/06/2022

Deep Learning-enabled Spatial Phase Unwrapping for 3D Measurement

In terms of 3D imaging speed and system cost, the single-camera system p...
research
04/23/2019

Replay attack detection with complementary high-resolution information using end-to-end DNN for the ASVspoof 2019 Challenge

In this study, we concentrate on replacing the process of extracting han...
research
04/13/2021

Aliasing is your Ally: End-to-End Super-Resolution from Raw Image Bursts

This presentation addresses the problem of reconstructing a high-resolut...
research
07/20/2020

Learning Adaptive Sampling and Reconstruction for Volume Visualization

A central challenge in data visualization is to understand which data sa...

Please sign up or login with your details

Forgot password? Click here to reset