Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

10/07/2016
by   Ashwin K Vijayakumar, et al.
0

Neural sequence models are widely used to model time-series data in many fields. Equally ubiquitous is the usage of beam search (BS) as an approximate inference algorithm to decode output sequences from these models. BS explores the search space in a greedy left-right fashion retaining only the top-B candidates -- resulting in sequences that differ only slightly from each other. Producing lists of nearly identical sequences is not only computationally wasteful but also typically fails to capture the inherent ambiguity of complex AI tasks. To overcome this problem, we propose Diverse Beam Search (DBS), an alternative to BS that decodes a list of diverse outputs by optimizing for a diversity-augmented objective. We observe that our method finds better top-1 solutions by controlling for the exploration and exploitation of the search space -- implying that DBS is a better search algorithm. Moreover, these gains are achieved with minimal computational or memory overhead as compared to beam search. To demonstrate the broad applicability of our method, we present results on image captioning, machine translation and visual question generation using both standard quantitative metrics and qualitative human studies. Our method consistently outperforms BS and previously proposed techniques for diverse decoding from neural sequence models.

READ FULL TEXT

page 13

page 14

research
05/24/2017

Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning

We develop the first approximate inference algorithm for 1-Best (and M-B...
research
09/17/2019

BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation

This study mainly investigates two decoding problems in neural keyphrase...
research
06/14/2021

Determinantal Beam Search

Beam search is a go-to strategy for decoding neural sequence models. The...
research
05/19/2023

Marginalized Beam Search Algorithms for Hierarchical HMMs

Inferring a state sequence from a sequence of measurements is a fundamen...
research
11/25/2016

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

In this paper, we propose a simple, fast decoding algorithm that fosters...
research
08/01/2017

A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models

Beam search is a desirable choice of test-time decoding algorithm for ne...
research
11/02/2018

Importance of a Search Strategy in Neural Dialogue Modelling

Search strategies for generating a response from a neural dialogue model...

Please sign up or login with your details

Forgot password? Click here to reset