Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels

11/07/2021
by   Zexi Hu, et al.
6

Despite the recent progress in light field super-resolution (LFSR) achieved by convolutional neural networks, the correlation information of light field (LF) images has not been sufficiently studied and exploited due to the complexity of 4D LF data. To cope with such high-dimensional LF data, most of the existing LFSR methods resorted to decomposing it into lower dimensions and subsequently performing optimization on the decomposed sub-spaces. However, these methods are inherently limited as they neglected the characteristics of the decomposition operations and only utilized a limited set of LF sub-spaces ending up failing to comprehensively extract spatio-angular features and leading to a performance bottleneck. To overcome these limitations, in this paper, we thoroughly discover the potentials of LF decomposition and propose a novel concept of decomposition kernels. In particular, we systematically unify the decomposition operations of various sub-spaces into a series of such decomposition kernels, which are incorporated into our proposed Decomposition Kernel Network (DKNet) for comprehensive spatio-angular feature extraction. The proposed DKNet is experimentally verified to achieve substantial improvements by 1.35 dB, 0.83 dB, and 1.80 dB PSNR in 2x, 3x and 4x LFSR scales, respectively, when compared with the state-of-the-art methods. To further improve DKNet in producing more visually pleasing LFSR results, based on the VGG network, we propose a LFVGG loss to guide the Texture-Enhanced DKNet (TE-DKNet) to generate rich authentic textures and enhance LF images' visual quality significantly. We also propose an indirect evaluation metric by taking advantage of LF material recognition to objectively assess the perceptual enhancement brought by the LFVGG loss.

READ FULL TEXT

page 1

page 2

page 7

page 8

page 10

page 11

research
07/25/2022

Sub-Aperture Feature Adaptation in Single Image Super-resolution Model for Light Field Imaging

With the availability of commercial Light Field (LF) cameras, LF imaging...
research
11/23/2019

Joint Spatial and Angular Super-Resolution from a Single Image

Synthesizing a densely sampled light field from a single image is highly...
research
01/12/2018

Light Field Super-Resolution using a Low-Rank Prior and Deep Convolutional Neural Networks

Light field imaging has recently known a regain of interest due to the a...
research
09/27/2018

A Simple Framework to Leverage State-Of-The-Art Single-Image Super-Resolution Methods to Restore Light Fields

Plenoptic cameras offer a cost effective solution to capture light field...
research
03/29/2020

High-Order Residual Network for Light Field Super-Resolution

Plenoptic cameras usually sacrifice the spatial resolution of their SAIs...
research
05/26/2020

Perceptual Extreme Super Resolution Network with Receptive Field Block

Perceptual Extreme Super-Resolution for single image is extremely diffic...
research
08/15/2016

Intrinsic Light Field Images

We present a method to automatically decompose a light field into its in...

Please sign up or login with your details

Forgot password? Click here to reset