DeepAI AI Chat
Log In Sign Up

Learning Layout and Style Reconfigurable GANs for Controllable Image Synthesis

03/25/2020
by   Wei Sun, et al.
11

With the remarkable recent progress on learning deep generative models, it becomes increasingly interesting to develop models for controllable image synthesis from reconfigurable inputs. This paper focuses on a recent emerged task, layout-to-image, to learn generative models that are capable of synthesizing photo-realistic images from spatial layout (i.e., object bounding boxes configured in an image lattice) and style (i.e., structural and appearance variations encoded by latent vectors). This paper first proposes an intuitive paradigm for the task, layout-to-mask-to-image, to learn to unfold object masks of given bounding boxes in an input layout to bridge the gap between the input layout and synthesized images. Then, this paper presents a method built on Generative Adversarial Networks for the proposed layout-to-mask-to-image with style control at both image and mask levels. Object masks are learned from the input layout and iteratively refined along stages in the generator network. Style control at the image level is the same as in vanilla GANs, while style control at the object mask level is realized by a proposed novel feature normalization scheme, Instance-Sensitive and Layout-Aware Normalization. In experiments, the proposed method is tested in the COCO-Stuff dataset and the Visual Genome dataset with state-of-the-art performance obtained.

READ FULL TEXT

page 8

page 9

page 12

page 13

page 14

page 15

page 16

page 17

08/20/2019

Image Synthesis From Reconfigurable Layout and Style

Despite remarkable recent progress on both unconditional and conditional...
03/15/2021

Deep Consensus Learning

Both generative learning and discriminative learning have recently witne...
03/04/2022

Interactive Image Synthesis with Panoptic Layout Generation

Interactive image synthesis from user-guided input is a challenging task...
12/29/2021

Learning Inception Attention for Image Synthesis and Image Recognition

Image synthesis and image recognition have witnessed remarkable progress...
12/25/2019

Controllable and Progressive Image Extrapolation

Image extrapolation aims at expanding the narrow field of view of a give...
03/22/2021

Context-Aware Layout to Image Generation with Enhanced Object Appearance

A layout to image (L2I) generation model aims to generate a complicated ...
03/16/2020

House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation

This paper proposes a novel graph-constrained generative adversarial net...

Code Repositories

ParametricSatelliteImageGeneration

None


view repo