A Novel Approach to Artistic Textual Visualization via GAN

by   Yichi Ma, et al.

While the visualization of statistical data tends to a mature technology, the visualization of textual data is still in its infancy, especially for the artistic text. Due to the fact that visualization of artistic text is valuable and attractive in both art and information science, we attempt to realize this tentative idea in this article. We propose the Generative Adversarial Network based Artistic Textual Visualization (GAN-ATV) which can create paintings after analyzing the semantic content of existing poems. Our GAN-ATV consists of two main sections: natural language analysis section and visual information synthesis section. In natural language analysis section, we use Bag-of-Word (BoW) feature descriptors and a two-layer network to mine and analyze the high-level semantic information from poems. In visual information synthesis section, we design a cross-modal semantic understanding module and integrate it with Generative Adversarial Network (GAN) to create paintings, whose content are corresponding to the original poems. Moreover, in order to train our GAN-ATV and verify its performance, we establish a cross-modal artistic dataset named "Cross-Art". In the Cross-Art dataset, there are six topics and each topic has their corresponding paintings and poems. The experimental results on Cross-Art dataset are shown in this article.


page 3

page 4

page 7


SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network

Cross-modal hashing aims to map heterogeneous multimedia data into a com...

Cross-Modal Contrastive Learning for Text-to-Image Generation

The output of text-to-image synthesis systems should be coherent, clear,...

SyncGAN: Synchronize the Latent Space of Cross-modal Generative Adversarial Networks

Generative adversarial network (GAN) has achieved impressive success on ...

Visual-Tactile Cross-Modal Data Generation using Residue-Fusion GAN with Feature-Matching and Perceptual Losses

Existing psychophysical studies have revealed that the cross-modal visua...

Visual Answer Localization with Cross-modal Mutual Knowledge Transfer

The goal of visual answering localization (VAL) in the video is to obtai...

Product Title Refinement via Multi-Modal Generative Adversarial Learning

Nowadays, an increasing number of customers are in favor of using E-comm...

Please sign up or login with your details

Forgot password? Click here to reset