Towards Diverse and Accurate Image Captions via Reinforcing Determinantal Point Process

08/14/2019
by   Qingzhong Wang, et al.
0

Although significant progress has been made in the field of automatic image captioning, it is still a challenging task. Previous works normally pay much attention to improving the quality of the generated captions but ignore the diversity of captions. In this paper, we combine determinantal point process (DPP) and reinforcement learning (RL) and propose a novel reinforcing DPP (R-DPP) approach to generate a set of captions with high quality and diversity for an image. We show that R-DPP performs better on accuracy and diversity than using noise as a control signal (GANs, VAEs). Moreover, R-DPP is able to preserve the modes of the learned distribution. Hence, beam search algorithm can be applied to generate a single accurate caption, which performs better than other RL-based models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/27/2020

Analysis of diversity-accuracy tradeoff in image captioning

We investigate the effect of different model architectures, training obj...
research
03/28/2019

Describing like humans: on diversity in image captioning

Recently, the state-of-the-art models for image captioning have overtake...
research
06/08/2018

Dank Learning: Generating Memes Using Deep Neural Networks

We introduce a novel meme generation system, which given any image can p...
research
12/06/2022

Switching to Discriminative Image Captioning by Relieving a Bottleneck of Reinforcement Learning

Discriminativeness is a desirable feature of image captions: captions sh...
research
05/28/2022

Variational Transformer: A Framework Beyond the Trade-off between Accuracy and Diversity for Image Captioning

Accuracy and Diversity are two essential metrizable manifestations in ge...
research
08/02/2023

ADS-Cap: A Framework for Accurate and Diverse Stylized Captioning with Unpaired Stylistic Corpora

Generating visually grounded image captions with specific linguistic sty...
research
08/23/2011

Artificial Neural Network and Rough Set for HV Bushings Condition Monitoring

Most transformer failures are attributed to bushings failures. Hence it ...

Please sign up or login with your details

Forgot password? Click here to reset