3D scene reconstruction from monocular spherical video with motion parallax

06/14/2022
by   Kenji Tanaka, et al.
0

In this paper, we describe a method to capture nearly entirely spherical (360 degree) depth information using two adjacent frames from a single spherical video with motion parallax. After illustrating a spherical depth information retrieval using two spherical cameras, we demonstrate monocular spherical stereo by using stabilized first-person video footage. Experiments demonstrated that the depth information was retrieved on up to 97 solid angle. At a speed of 30 km/h, we were able to estimate the depth of an object located over 30 m from the camera. We also reconstructed the 3D structures (point cloud) using the obtained depth data and confirmed the structures can be clearly observed. We can apply this method to 3D structure retrieval of surrounding environments such as 1) previsualization, location hunting/planning of a film, 2) real scene/computer graphics synthesis and 3) motion capture. Thanks to its simplicity, this method can be applied to various videos. As there is no pre-condition other than to be a 360 video with motion parallax, we can use any 360 videos including those on the Internet to reconstruct the surrounding environments. The cameras can be lightweight enough to be mounted on a drone. We also demonstrated such applications.

READ FULL TEXT

page 4

page 5

page 7

page 8

page 9

page 10

page 11

research
04/10/2019

Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras

We present a novel method for simultaneous learning of depth, egomotion,...
research
04/24/2023

Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis

We explore the task of embodied view synthesis from monocular videos of ...
research
04/25/2019

Learning the Depths of Moving People by Watching Frozen People

We present a method for predicting dense depth in scenarios where both a...
research
04/26/2021

Spherical formulation of geometric motion segmentation constraints in fisheye cameras

We introduce a visual motion segmentation method employing spherical geo...
research
06/22/2023

3D Reconstruction of Spherical Images based on Incremental Structure from Motion

3D reconstruction plays an increasingly important role in modern photogr...
research
03/30/2021

Endo-Depth-and-Motion: Localization and Reconstruction in Endoscopic Videos using Depth Networks and Photometric Constraints

Estimating a scene reconstruction and the camera motion from in-body vid...
research
03/21/2019

Quantitative Depth Quality Assessment of RGBD Cameras At Close Range Using 3D Printed Fixtures

Mobile robots that manipulate their environments require high-accuracy s...

Please sign up or login with your details

Forgot password? Click here to reset