360^∘ Stereo Image Composition with Depth Adaption

12/20/2022
by   Kun Huang, et al.
0

360^∘ images and videos have become an economic and popular way to provide VR experiences using real-world content. However, the manipulation of the stereo panoramic content remains less explored. In this paper, we focus on the 360^∘ image composition problem, and develop a solution that can take an object from a stereo image pair and insert it at a given 3D position in a target stereo panorama, with well-preserved geometry information. Our method uses recovered 3D point clouds to guide the composited image generation. More specifically, we observe that using only a one-off operation to insert objects into equirectangular images will never produce satisfactory depth perception and generate ghost artifacts when users are watching the result from different view directions. Therefore, we propose a novel view-dependent projection method that segments the object in 3D spherical space with the stereo camera pair facing in that direction. A deep depth densification network is proposed to generate depth guidance for the stereo image generation of each view segment according to the desired position and pose of the inserted object. We finally merge the synthesized view segments and blend the objects into the target stereo 360^∘ scene. A user study demonstrates that our method can provide good depth perception and removes ghost artifacts. The view-dependent solution is a potential paradigm for other content manipulation methods for 360^∘ images and videos.

READ FULL TEXT

page 1

page 3

page 4

page 6

page 7

page 8

page 9

page 10

research
11/03/2022

StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS

Most existing methods for category-level pose estimation rely on object ...
research
02/22/2023

MVTrans: Multi-View Perception of Transparent Objects

Transparent object perception is a crucial skill for applications such a...
research
02/27/2022

Next-Best-View Prediction for Active Stereo Cameras and Highly Reflective Objects

Depth acquisition with the active stereo camera is a challenging task fo...
research
03/11/2021

Unknown Object Segmentation from Stereo Images

Although instance-aware perception is a key prerequisite for many autono...
research
02/09/2019

Depth-Map Generation using Pixel Matching in Stereoscopic Pair of Images

Modern day multimedia content generation and dissemination is moving tow...
research
11/28/2013

Glasgow's Stereo Image Database of Garments

To provide insight into cloth perception and manipulation with an active...
research
09/23/2021

A Learned Stereo Depth System for Robotic Manipulation in Homes

We present a passive stereo depth system that produces dense and accurat...

Please sign up or login with your details

Forgot password? Click here to reset