NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting

07/26/2021
by   Tiancheng Sun, et al.
10

Human portraits exhibit various appearances when observed from different views under different lighting conditions. We can easily imagine how the face will look like in another setup, but computer algorithms still fail on this problem given limited observations. To this end, we present a system for portrait view synthesis and relighting: given multiple portraits, we use a neural network to predict the light-transport field in 3D space, and from the predicted Neural Light-transport Field (NeLF) produce a portrait from a new camera view under a new environmental lighting. Our system is trained on a large number of synthetic models, and can generalize to different synthetic and real portraits under various lighting conditions. Our method achieves simultaneous view synthesis and relighting given multi-view portraits as the input, and achieves state-of-the-art results.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 6

page 8

page 10

page 11

research
08/09/2020

Neural Light Transport for Relighting and View Synthesis

The light transport (LT) of a scene describes how it appears under diffe...
research
02/02/2022

Eikonal Fields for Refractive Novel-View Synthesis

We tackle the problem of generating novel-view images from collections o...
research
06/18/2019

DeepView: View Synthesis with Learned Gradient Descent

We present a novel approach to view synthesis using multiplane images (M...
research
08/13/2018

Vision-Based Preharvest Yield Mapping for Apple Orchards

We present an end-to-end computer vision system for mapping yield in an ...
research
03/20/2023

ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning

Although many recent works have investigated generalizable NeRF-based no...
research
12/17/2020

Relightable 3D Head Portraits from a Smartphone Video

In this work, a system for creating a relightable 3D portrait of a human...
research
06/21/2021

Moving in a 360 World: Synthesizing Panoramic Parallaxes from a Single Panorama

We present Omnidirectional Neural Radiance Fields (OmniNeRF), the first ...

Please sign up or login with your details

Forgot password? Click here to reset