DeepAI AI Chat
Log In Sign Up

Learning What and Where to Draw

10/08/2016
by   Scott Reed, et al.
Google
University of Michigan
Max Planck Society
0

Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location. We show high-quality 128 x 128 image synthesis on the Caltech-UCSD Birds dataset, conditioned on both informal text descriptions and also object location. Our system exposes control over both the bounding box around the bird and its constituent parts. By modeling the conditional distributions over part locations, our system also enables conditioning on arbitrary subsets of parts (e.g. only the beak and tail), yielding an efficient interface for picking part locations. We also show preliminary results on the more challenging domain of text- and location-controllable synthesis of images of human actions on the MPII Human Pose dataset.

READ FULL TEXT

page 2

page 6

page 7

page 8

05/01/2020

Adversarial Synthesis of Human Pose from Text

This work introduces the novel task of human pose synthesis from text. I...
05/17/2016

Generative Adversarial Text to Image Synthesis

Automatic synthesis of realistic images from text would be interesting a...
07/30/2018

Pose Guided Human Video Generation

Due to the emergence of Generative Adversarial Networks, video synthesis...
02/18/2019

Generative Adversarial Networks Synthesize Realistic OCT Images of the Retina

We report, to our knowledge, the first end-to-end application of Generat...
12/01/2016

Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts

Automatic image synthesis research has been rapidly growing with deep ne...
06/07/2021

PcDGAN: A Continuous Conditional Diverse Generative Adversarial Network For Inverse Design

Engineering design tasks often require synthesizing new designs that mee...
09/09/2023

VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis

Unsupervised learning of 3D-aware generative adversarial networks has la...