State of the Art on Neural Rendering

by   Ayush Tewari, et al.

Efficient rendering of photo-realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo-realistic images from hand-crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo-realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state-of-the-art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photo-realistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. This state-of-the-art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.


page 1

page 8

page 10

page 12

page 14

page 15

page 16

page 18


Deep Graphics Encoder for Real-Time Video Makeup Synthesis from Example

While makeup virtual-try-on is now widespread, parametrizing a computer ...

Zahir: a Object-Oriented Framework for Computer Graphics

In this article we present Zahir, a framework for experimentation in Com...

Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination

Given a set of images of a scene, the re-rendering of this scene from no...

Blending Generative Adversarial Image Synthesis with Rendering for Computer Graphics

Conventional computer graphics pipelines require detailed 3D models, mes...

Deferred Neural Rendering: Image Synthesis using Neural Textures

The modern computer graphics pipeline can synthesize images at remarkabl...

Neural Radiance Fields: Past, Present, and Future

The various aspects like modeling and interpreting 3D environments and s...

A Novel Method for Vectorization

Vectorization of images is a key concern uniting computer graphics and c...

Code Repositories


機械学習、バイオインフォマティクスのツールや論文、ニュースなど。最終更新 2020/4/25

view repo

Please sign up or login with your details

Forgot password? Click here to reset