Deep 3D-Zoom Net: Unsupervised Learning of Photo-Realistic 3D-Zoom

09/20/2019
by   Juan Luis Gonzalez Bello, et al.
5

The 3D-zoom operation is the positive translation of the camera in the Z-axis, perpendicular to the image plane. In contrast, the optical zoom changes the focal length and the digital zoom is used to enlarge a certain region of an image to the original image size. In this paper, we are the first to formulate an unsupervised 3D-zoom learning problem where images with an arbitrary zoom factor can be generated from a given single image. An unsupervised framework is convenient, as it is a challenging task to obtain a 3D-zoom dataset of natural scenes due to the need for special equipment to ensure camera movement is restricted to the Z-axis. In addition, the objects in the scenes should not move when being captured, which hinders the construction of a large dataset of outdoor scenes. We present a novel unsupervised framework to learn how to generate arbitrarily 3D-zoomed versions of a single image, not requiring a 3D-zoom ground truth, called the Deep 3D-Zoom Net. The Deep 3D-Zoom Net incorporates the following features: (i) transfer learning from a pre-trained disparity estimation network via a back re-projection reconstruction loss; (ii) a fully convolutional network architecture that models depth-image-based rendering (DIBR), taking into account high-frequency details without the need for estimating the intermediate disparity; and (iii) incorporating a discriminator network that acts as a no-reference penalty for unnaturally rendered areas. Even though there is no baseline to fairly compare our results, our method outperforms previous novel view synthesis research in terms of realistic appearance on large camera baselines. We performed extensive experiments to verify the effectiveness of our method on the KITTI and Cityscapes datasets.

READ FULL TEXT

page 4

page 6

page 7

page 8

research
10/02/2019

Deep 3D Pan via adaptive "t-shaped" convolutions with global and local adaptive dilations

Recent advances in deep learning have shown promising results in many lo...
research
03/20/2019

A Novel Monocular Disparity Estimation Network with Domain Transformation and Ambiguity Learning

Convolutional neural networks (CNN) have shown state-of-the-art results ...
research
09/03/2019

Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus

Unsupervised learning based depth estimation methods have received more ...
research
12/15/2019

DerainCycleGAN: An Attention-guided Unsupervised Benchmark for Single Image Deraining and Rainmaking

Single image deraining (SID) is an important and challenging topic in em...
research
04/06/2020

Appearance Shock Grammar for Fast Medial Axis Extraction from Real Images

We combine ideas from shock graph theory with more recent appearance-bas...
research
10/04/2021

HDR-cGAN: Single LDR to HDR Image Translation using Conditional GAN

The prime goal of digital imaging techniques is to reproduce the realist...
research
09/16/2019

Learning Geo-Temporal Image Features

We propose to implicitly learn to extract geo-temporal image features, w...

Please sign up or login with your details

Forgot password? Click here to reset