GNeRF: GAN-based Neural Radiance Field without Posed Camera

03/29/2021
by   Quan Meng, et al.
21

We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field reconstruction for the complex scenarios with unknown and even randomly initialized camera poses. Recent NeRF-based advances have gained popularity for remarkable realistic novel view synthesis. However, most of them heavily rely on accurate camera poses estimation, while few recent methods can only optimize the unknown camera poses in roughly forward-facing scenes with relatively short camera trajectories and require rough camera poses initialization. Differently, our GNeRF only utilizes randomly initialized poses for complex outside-in scenarios. We propose a novel two-phases end-to-end framework. The first phase takes the use of GANs into the new realm for coarse camera poses and radiance fields jointly optimization, while the second phase refines them with additional photometric loss. We overcome local minima using a hybrid and iterative optimization scheme. Extensive experiments on a variety of synthetic and natural scenes demonstrate the effectiveness of GNeRF. More impressively, our approach outperforms the baselines favorably in those scenes with repeated patterns or even low textures that are regarded as extremely challenging before.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

04/13/2021

BARF: Bundle-Adjusting Neural Radiance Fields

Neural Radiance Fields (NeRF) have recently gained a surge of interest w...
02/14/2021

NeRF–: Neural Radiance Fields Without Known Camera Parameters

This paper tackles the problem of novel view synthesis (NVS) from 2D ima...
04/12/2022

GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimation

Despite Neural Radiance Fields (NeRF) showing compelling results in phot...
07/22/2022

InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images

We present a method for learning to generate unbounded flythrough videos...
07/06/2022

VMRF: View Matching Neural Radiance Fields

Neural Radiance Fields (NeRF) have demonstrated very impressive performa...
12/16/2020

SimuGAN: Unsupervised forward modeling and optimal design of a LIDAR Camera

Energy-saving LIDAR camera for short distances estimates an object's dis...