DeepAI AI Chat
Log In Sign Up

Learning What and Where to Draw

by   Scott Reed, et al.
University of Michigan
Max Planck Society

Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location. We show high-quality 128 x 128 image synthesis on the Caltech-UCSD Birds dataset, conditioned on both informal text descriptions and also object location. Our system exposes control over both the bounding box around the bird and its constituent parts. By modeling the conditional distributions over part locations, our system also enables conditioning on arbitrary subsets of parts (e.g. only the beak and tail), yielding an efficient interface for picking part locations. We also show preliminary results on the more challenging domain of text- and location-controllable synthesis of images of human actions on the MPII Human Pose dataset.


page 2

page 6

page 7

page 8


Adversarial Synthesis of Human Pose from Text

This work introduces the novel task of human pose synthesis from text. I...

Generative Adversarial Text to Image Synthesis

Automatic synthesis of realistic images from text would be interesting a...

Pose Guided Human Video Generation

Due to the emergence of Generative Adversarial Networks, video synthesis...

Generative Adversarial Networks Synthesize Realistic OCT Images of the Retina

We report, to our knowledge, the first end-to-end application of Generat...

Learning to Generate Images of Outdoor Scenes from Attributes and Semantic Layouts

Automatic image synthesis research has been rapidly growing with deep ne...

PcDGAN: A Continuous Conditional Diverse Generative Adversarial Network For Inverse Design

Engineering design tasks often require synthesizing new designs that mee...

VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis

Unsupervised learning of 3D-aware generative adversarial networks has la...