Object-Centric Image Generation from Layouts

03/16/2020
by   Tristan Sylvain, et al.
7

Despite recent impressive results on single-object and single-domain image generation, the generation of complex scenes with multiple objects remains challenging. In this paper, we start with the idea that a model must be able to understand individual objects and relationships between objects in order to generate complex scenes well. Our layout-to-image-generation method, which we call Object-Centric Generative Adversarial Network (or OC-GAN), relies on a novel Scene-Graph Similarity Module (SGSM). The SGSM learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We also propose changes to the conditioning mechanism of the generator that enhance its object instance-awareness. Apart from improving image quality, our contributions mitigate two failure modes in previous approaches: (1) spurious objects being generated without corresponding bounding boxes in the layout, and (2) overlapping bounding boxes in the layout leading to merged objects in images. Extensive quantitative evaluation and ablation studies demonstrate the impact of our contributions, with our model outperforming previous state-of-the-art approaches on both the COCO-Stuff and Visual Genome datasets. Finally, we address an important limitation of evaluation metrics used in previous works by introducing SceneFID – an object-centric adaptation of the popular Fréchet Inception Distance metric, that is better suited for multi-object images.

READ FULL TEXT

page 2

page 11

page 24

page 25

research
08/27/2020

Attribute-guided image generation from layout

Recent approaches have achieved great success in image generation from s...
research
04/05/2022

DT2I: Dense Text-to-Image Generation from Region Descriptions

Despite astonishing progress, generating realistic images of complex sce...
research
01/03/2019

Generating Multiple Objects at Spatially Distinct Locations

Recent improvements to Generative Adversarial Networks (GANs) have made ...
research
03/08/2023

Transformer-based Image Generation from Scene Graphs

Graph-structured scene descriptions can be efficiently used in generativ...
research
03/16/2020

House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation

This paper proposes a novel graph-constrained generative adversarial net...
research
05/06/2023

Multi-object Video Generation from Single Frame Layouts

In this paper, we study video synthesis with emphasis on simplifying the...
research
03/25/2021

AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style

Conditional image synthesis from layout has recently attracted much inte...

Please sign up or login with your details

Forgot password? Click here to reset