Semantic Palette: Guiding Scene Generation with Class Proportions

06/03/2021
by   Guillaume Le Moing, et al.
6

Despite the recent progress of generative adversarial networks (GANs) at synthesizing photo-realistic images, producing complex urban scenes remains a challenging problem. Previous works break down scene generation into two consecutive phases: unconditional semantic layout synthesis and image synthesis conditioned on layouts. In this work, we propose to condition layout generation as well for higher semantic control: given a vector of class proportions, we generate layouts with matching composition. To this end, we introduce a conditional framework with novel architecture designs and learning objectives, which effectively accommodates class proportions to guide the scene generation process. The proposed architecture also allows partial layout editing with interesting applications. Thanks to the semantic control, we can produce layouts close to the real distribution, helping enhance the whole scene generation process. On different metrics and urban scene benchmarks, our models outperform existing baselines. Moreover, we demonstrate the merit of our approach for data augmentation: semantic segmenters trained on real layout-image pairs along with additional ones generated by our approach outperform models only trained on real pairs.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 6

page 8

page 13

page 14

page 15

page 16

page 17

11/26/2019

Semantic Bottleneck Scene Generation

Coupling the high-fidelity generation capabilities of label-conditional ...
12/01/2016

Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts

Automatic image synthesis research has been rapidly growing with deep ne...
07/01/2020

Future Urban Scenes Generation Through Vehicles Synthesis

In this work we propose a deep learning pipeline to predict the visual f...
07/19/2018

Compositional GAN: Learning Conditional Image Composition

Generative Adversarial Networks (GANs) can produce images of surprising ...
08/04/2021

Neural Scene Decoration from a Single Photograph

Furnishing and rendering an indoor scene is a common but tedious task fo...
11/21/2019

Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis

Despite the success of Generative Adversarial Networks (GANs) in image s...
12/10/2021

Roominoes: Generating Novel 3D Floor Plans From Existing 3D Rooms

Realistic 3D indoor scene datasets have enabled significant recent progr...

Code Repositories

SemanticPalette

Semantic Palette: Guiding Scene Generation with Class Proportions


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.