Visual Information Guided Zero-Shot Paraphrase Generation

01/22/2022
by   Zhe Lin, et al.
0

Zero-shot paraphrase generation has drawn much attention as the large-scale high-quality paraphrase corpus is limited. Back-translation, also known as the pivot-based method, is typical to this end. Several works leverage different information as "pivot" such as language, semantic representation and so on. In this paper, we explore using visual information such as image as the "pivot" of back-translation. Different with the pipeline back-translation method, we propose visual information guided zero-shot paraphrase generation (ViPG) based only on paired image-caption data. It jointly trains an image captioning model and a paraphrasing model and leverage the image captioning model to guide the training of the paraphrasing model. Both automatic evaluation and human evaluation show our model can generate paraphrase with good relevancy, fluency and diversity, and image is a promising kind of pivot for zero-shot paraphrase generation.

READ FULL TEXT
research
11/13/2022

Large-Scale Bidirectional Training for Zero-Shot Image Captioning

When trained on large-scale datasets, image captioning models can unders...
research
03/06/2023

DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training

Large-scale pre-trained multi-modal models (e.g., CLIP) demonstrate stro...
research
08/13/2021

Detection and Captioning with Unseen Object Classes

Image caption generation is one of the most challenging problems at the ...
research
02/19/2018

Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types

We present a neural model for question generation from knowledge base tr...
research
11/07/2018

Learning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning

Although promising results have been achieved in video captioning, exist...
research
04/18/2022

Cross-view Brain Decoding

How the brain captures the meaning of linguistic stimuli across multiple...
research
05/21/2023

Contrastive Language-Image Pretrained Models are Zero-Shot Human Scanpath Predictors

Understanding the mechanisms underlying human attention is a fundamental...

Please sign up or login with your details

Forgot password? Click here to reset