Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

11/26/2019
by   Osaid Rehman Nasir, et al.
19

Powerful generative adversarial networks (GAN) have been developed to automatically synthesize realistic images from text. However, most existing tasks are limited to generating simple images such as flowers from captions. In this work, we extend this problem to the less addressed domain of face generation from fine-grained textual descriptions of face, e.g., "A person has curly hair, oval face, and mustache". We are motivated by the potential of automated face generation to impact and assist critical tasks such as criminal face reconstruction. Since current datasets for the task are either very small or do not contain captions, we generate captions for images in the CelebA dataset by creating an algorithm to automatically convert a list of attributes to a set of captions. We then model the highly multi-modal problem of text to face generation as learning the conditional distribution of faces (conditioned on text) in same latent space. We utilize the current state-of-the-art GAN (DC-GAN with GAN-CLS loss) for learning conditional multi-modality. The presence of more fine-grained details and variable length of the captions makes the problem easier for a user but more difficult to handle compared to the other text-to-image tasks. We flipped the labels for real and fake images and added noise in discriminator. Generated images for diverse textual descriptions show promising results. In the end, we show how the widely used inceptions score is not a good metric to evaluate the performance of generative models used for synthesizing faces from text.

READ FULL TEXT

page 1

page 3

page 7

research
01/22/2023

Face Generation from Textual Features using Conditionally Trained Inputs to Generative Adversarial Networks

Generative Networks have proved to be extremely effective in image resto...
research
08/31/2023

Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images

Generating 3D faces from textual descriptions has a multitude of applica...
research
07/29/2023

Sat2Cap: Mapping Fine-Grained Textual Descriptions from Satellite Images

We propose a novel weakly supervised approach for creating maps using fr...
research
05/03/2017

FOIL it! Find One mismatch between Image and Language caption

In this paper, we aim to understand whether current language and vision ...
research
09/14/2023

Looking at words and points with attention: a benchmark for text-to-shape coherence

While text-conditional 3D object generation and manipulation have seen r...
research
03/10/2018

Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions

The past few years have witnessed renewed interest in NLP tasks at the i...
research
06/06/2021

MOC-GAN: Mixing Objects and Captions to Generate Realistic Images

Generating images with conditional descriptions gains increasing interes...

Please sign up or login with your details

Forgot password? Click here to reset