DeepAI AI Chat
Log In Sign Up

Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis

04/04/2021
by   Yaxuan Zhu, et al.
12

How to effectively represent camera pose is an essential problem in 3D computer vision, especially in tasks such as camera pose regression and novel view synthesis. Traditionally, 3D position of the camera is represented by Cartesian coordinate and the orientation is represented by Euler angle or quaternions. These representations are manually designed, which may not be the most effective representation for downstream tasks. In this work, we propose an approach to learn neural representations of camera poses and 3D scenes, coupled with neural representations of local camera movements. Specifically, the camera pose and 3D scene are represented as vectors and the local camera movement is represented as a matrix operating on the vector of the camera pose. We demonstrate that the camera movement can further be parametrized by a matrix Lie algebra that underlies a rotation system in the neural space. The vector representations are then concatenated and generate the posed 2D image through a decoder network. The model is learned from only posed 2D images and corresponding camera poses, without access to depths or shapes. We conduct extensive experiments on synthetic and real datasets. The results show that compared with other camera pose representations, our learned representation is more robust to noise in novel view synthesis and more effective in camera pose regression.

READ FULL TEXT

page 6

page 13

page 14

page 15

page 16

page 17

page 18

page 19

07/06/2022

VMRF: View Matching Neural Radiance Fields

Neural Radiance Fields (NeRF) have demonstrated very impressive performa...
04/13/2021

BARF: Bundle-Adjusting Neural Radiance Fields

Neural Radiance Fields (NeRF) have recently gained a surge of interest w...
02/14/2021

NeRF–: Neural Radiance Fields Without Known Camera Parameters

This paper tackles the problem of novel view synthesis (NVS) from 2D ima...
10/13/2021

LENS: Localization enhanced by NeRF synthesis

Neural Radiance Fields (NeRF) have recently demonstrated photo-realistic...
02/24/2018

Euler angles based loss function for camera relocalization with Deep learning

Deep learning has been applied to camera relocalization, in particular, ...
12/06/2019

Gaussian Process Priors for View-Aware Inference

We derive a principled framework for encoding prior knowledge of informa...
12/02/2018

ECO: Egocentric Cognitive Mapping

We present a new method to localize a camera within a previously unseen ...