DeepAI AI Chat
Log In Sign Up

Text-to-Image Generation with Attention Based Recurrent Neural Networks

by   Tehseen Zia, et al.

Conditional image modeling based on textual descriptions is a relatively new domain in unsupervised learning. Previous approaches use a latent variable model and generative adversarial networks. While the formers are approximated by using variational auto-encoders and rely on the intractable inference that can hamper their performance, the latter is unstable to train due to Nash equilibrium based objective function. We develop a tractable and stable caption-based image generation model. The model uses an attention-based encoder to learn word-to-pixel dependencies. A conditional autoregressive based decoder is used for learning pixel-to-pixel dependencies and generating images. Experimentations are performed on Microsoft COCO, and MNIST-with-captions datasets and performance is evaluated by using the Structural Similarity Index. Results show that the proposed model performs better than contemporary approaches and generate better quality images. Keywords: Generative image modeling, autoregressive image modeling, caption-based image generation, neural attention, recurrent neural networks.


page 6

page 8


Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks

Recently, realistic image generation using deep neural networks has beco...

Language Generation with Recurrent Generative Adversarial Networks without Pre-training

Generative Adversarial Networks (GANs) have shown great promise recently...

DRAW: A Recurrent Neural Network For Image Generation

This paper introduces the Deep Recurrent Attentive Writer (DRAW) neural ...

Vector Learning for Cross Domain Representations

Recently, generative adversarial networks have gained a lot of popularit...

Pixel Recurrent Neural Networks

Modeling the distribution of natural images is a landmark problem in uns...

Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation

A 360-degree (omni-directional) image provides an all-encompassing spher...

Cycle Text-To-Image GAN with BERT

We explore novel approaches to the task of image generation from their r...