Neural View-Interpolation for Sparse Light Field Video

10/30/2019
by   Mojtaba Bemana, et al.
19

We suggest representing light field (LF) videos as "one-off" neural networks (NN), i.e., a learned mapping from view-plus-time coordinates to high-resolution color values, trained on sparse views. Initially, this sounds like a bad idea for three main reasons: First, a NN LF will likely have less quality than a same-sized pixel basis representation. Second, only few training data, e.g., 9 exemplars per frame are available for sparse LF videos. Third, there is no generalization across LFs, but across view and time instead. Consequently, a network needs to be trained for each LF video. Surprisingly, these problems can turn into substantial advantages: Other than the linear pixel basis, a NN has to come up with a compact, non-linear i.e., more intelligent, explanation of color, conditioned on the sparse view and time coordinates. As observed for many NN however, this representation now is interpolatable: if the image output for sparse view coordinates is plausible, it is for all intermediate, continuous coordinates as well. Our specific network architecture involves a differentiable occlusion-aware warping step, which leads to a compact set of trainable parameters and consequently fast learning and fast execution.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 9

page 11

research
10/30/2019

Neural View-Interpolation for Sparse LightField Video

We suggest representing light field (LF) videos as "one-off" neural netw...
research
10/01/2020

X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation

We suggest to represent an X-Field -a set of 2D images taken across diff...
research
11/30/2021

Light Field Implicit Representation for Flexible Resolution Reconstruction

Inspired by the recent advances in implicitly representing signals with ...
research
08/12/2020

Self-supervised Light Field View Synthesis Using Cycle Consistency

High angular resolution is advantageous for practical applications of li...
research
05/29/2018

CocoNet: A deep neural network for mapping pixel coordinates to color values

In this paper, we propose a deep neural network approach for mapping the...
research
03/30/2023

Implicit View-Time Interpolation of Stereo Videos using Multi-Plane Disparities and Non-Uniform Coordinates

In this paper, we propose an approach for view-time interpolation of ste...

Please sign up or login with your details

Forgot password? Click here to reset