Image Generation from Layout

11/28/2018
by   Bo Zhao, et al.
0

Despite significant recent progress on generative models, controlled generation of images depicting multiple and complex object layouts is still a difficult problem. Among the core challenges are the diversity of appearance a given object may possess and, as a result, exponential set of images consistent with a specified layout. To address these challenges, we propose a novel approach for layout-based image generation; we call it Layout2Im. Given the coarse spatial layout (bounding boxes + object categories), our model can generate a set of realistic images which have the correct objects in the desired locations. The representation of each object is disentangled into a specified/certain part (category) and an unspecified/uncertain part (appearance). The category is encoded using a word embedding and the appearance is distilled into a low-dimensional vector sampled from a normal distribution. Individual object representations are composed together using convolutional LSTM, to obtain an encoding of the complete layout, and then decoded to an image. Several loss terms are introduced to encourage accurate and diverse generation. The proposed Layout2Im model significantly outperforms the previous state of the art, boosting the best reported inception score by 24.66 28.57 respectively. Extensive experiments also demonstrate our method's ability to generate complex and diverse images with multiple objects.

READ FULL TEXT

page 3

page 6

page 7

research
08/27/2020

Attribute-guided image generation from layout

Recent approaches have achieved great success in image generation from s...
research
07/06/2021

DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis

Despite significant progress on current state-of-the-art image generatio...
research
08/28/2020

Person-in-Context Synthesiswith Compositional Structural Space

Despite significant progress, controlled generation of complex images wi...
research
05/30/2020

OPAL-Net: A Generative Model for Part-based Object Layout Generation

We propose OPAL-Net, a novel hierarchical architecture for part-based la...
research
04/30/2022

LayoutBERT: Masked Language Layout Model for Object Insertion

Image compositing is one of the most fundamental steps in creative workf...
research
03/25/2021

AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Style

Conditional image synthesis from layout has recently attracted much inte...
research
02/27/2019

Object-driven Text-to-Image Synthesis via Adversarial Training

In this paper, we propose Object-driven Attentive Generative Adversarial...

Please sign up or login with your details

Forgot password? Click here to reset