Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video with Applications for Virtual Reality

10/14/2020
by   Alisha Sharma, et al.
0

We introduce a convolutional neural network model for unsupervised learning of depth and ego-motion from cylindrical panoramic video. Panoramic depth estimation is an important technology for applications such as virtual reality, 3D modeling, and autonomous robotic navigation. In contrast to previous approaches for applying convolutional neural networks to panoramic imagery, we use the cylindrical panoramic projection which allows for the use of the traditional CNN layers such as convolutional filters and max pooling without modification. Our evaluation of synthetic and real data shows that unsupervised learning of depth and ego-motion on cylindrical panoramic images can produce high-quality depth maps and that an increased field-of-view improves ego-motion estimation accuracy. We create two new datasets to evaluate our approach: a synthetic dataset created using the CARLA simulator, and Headcam, a novel dataset of panoramic video collected from a helmet-mounted camera while biking in an urban setting. We also apply our network to the problem of converting monocular panoramas to stereo panoramas.

READ FULL TEXT

page 2

page 4

page 5

page 8

page 10

page 11

page 12

research
01/04/2019

Unsupervised Learning of Depth and Ego-Motion from Panoramic Video

We introduce a convolutional neural network model for unsupervised learn...
research
04/01/2021

Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Masks

A new unsupervised learning method of depth and ego-motion using multipl...
research
06/09/2023

Circular Rectifiction of 3D Video and Efficient Modification of 3D-HEVC

Video acquired from multiple cameras located along a line is often recti...
research
04/13/2016

Deep3D: Fully Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks

As 3D movie viewing becomes mainstream and Virtual Reality (VR) market e...
research
02/15/2018

Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints

We present a novel approach for unsupervised learning of depth and ego-m...
research
04/05/2022

Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows

Existing approaches for Structure from Motion (SfM) produce impressive 3...
research
05/24/2017

Unsupervised Learning Layers for Video Analysis

This paper presents two unsupervised learning layers (UL layers) for lab...

Please sign up or login with your details

Forgot password? Click here to reset