Enhancing Photorealism Enhancement

by   Stephan R. Richter, et al.

We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new strategy for sampling image patches during training. We also introduce multiple architectural improvements in the deep network modules used for photorealism enhancement. We confirm the benefits of our contributions in controlled experiments and report substantial gains in stability and realism in comparison to recent image-to-image translation methods and a variety of other baselines.


page 1

page 4

page 5

page 6

page 7

page 10

page 11

page 15


Biphasic Learning of GANs for High-Resolution Image-to-Image Translation

Despite that the performance of image-to-image translation has been sign...

Speech Enhancement via Deep Spectrum Image Translation Network

Quality and intelligibility of speech signals are degraded under additiv...

Explaining Image Enhancement Black-Box Methods through a Path Planning Based Algorithm

Nowadays, image-to-image translation methods, are the state of the art f...

Enhancing Traffic Scene Predictions with Generative Adversarial Networks

We present a new two-stage pipeline for predicting frames of traffic sce...

Palette: Image-to-Image Diffusion Models

We introduce Palette, a simple and general framework for image-to-image ...

Multi-Domain Image-to-Image Translation with Adaptive Inference Graph

In this work, we address the problem of multi-domain image-to-image tran...