State of the Art on Neural Rendering

04/08/2020
by   Ayush Tewari, et al.
13

Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photo-realistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. This state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.

READ FULL TEXT

page 1

page 8

page 10

page 12

page 14

page 15

page 16

page 18

research
05/12/2021

Deep Graphics Encoder for Real-Time Video Makeup Synthesis from Example

While makeup virtual-try-on is now widespread, parametrizing a computer ...
research
09/07/2013

Zahir: a Object-Oriented Framework for Computer Graphics

In this article we present Zahir, a framework for experimentation in Com...
research
07/27/2022

Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination

Given a set of images of a scene, the re-rendering of this scene from no...
research
07/31/2020

Blending Generative Adversarial Image Synthesis with Rendering for Computer Graphics

Conventional computer graphics pipelines require detailed 3D models, mes...
research
04/28/2019

Deferred Neural Rendering: Image Synthesis using Neural Textures

The modern computer graphics pipeline can synthesize images at remarkabl...
research
04/20/2023

Neural Radiance Fields: Past, Present, and Future

The various aspects like modeling and interpreting 3D environments and s...
research
03/04/2014

A Novel Method for Vectorization

Vectorization of images is a key concern uniting computer graphics and c...

Please sign up or login with your details

Forgot password? Click here to reset