Novel View Video Prediction Using a Dual Representation

06/07/2021
by   Sarah Shiraz, et al.
0

We address the problem of novel view video prediction; given a set of input video clips from a single/multiple views, our network is able to predict the video from a novel view. The proposed approach does not require any priors and is able to predict the video from wider angular distances, upto 45 degree, as compared to the recent studies predicting small variations in viewpoint. Moreover, our method relies only onRGB frames to learn a dual representation which is used to generate the video from a novel viewpoint. The dual representation encompasses a view-dependent and a global representation which incorporates complementary details to enable novel view video prediction. We demonstrate the effectiveness of our framework on two real world datasets: NTU-RGB+D and CMU Panoptic. A comparison with the State-of-the-art novel view video prediction methods shows an improvement of 26.1 and 60

READ FULL TEXT

page 1

page 4

page 5

page 9

page 14

page 15

page 16

page 17

research
11/26/2018

Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning

The recent success in deep learning has lead to various effective repres...
research
10/29/2021

Novel View Synthesis from a Single Image via Unsupervised learning

View synthesis aims to generate novel views from one or more given sourc...
research
12/07/2021

ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints

Learning self-supervised video representation predominantly focuses on d...
research
05/04/2022

Video Extrapolation in Space and Time

Novel view synthesis (NVS) and video prediction (VP) are typically consi...
research
12/27/2021

Human View Synthesis using a Single Sparse RGB-D Input

Novel view synthesis for humans in motion is a challenging computer visi...
research
01/21/2023

Time-Conditioned Generative Modeling of Object-Centric Representations for Video Decomposition and Prediction

When perceiving the world from multiple viewpoints, humans have the abil...
research
11/29/2018

DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama

We present a deep learning framework, called DuLa-Net, to predict Manhat...

Please sign up or login with your details

Forgot password? Click here to reset