Determinantal Beam Search

06/14/2021
by   Clara Meister, et al.
0

Beam search is a go-to strategy for decoding neural sequence models. The algorithm can naturally be viewed as a subset optimization problem, albeit one where the corresponding set function does not reflect interactions between candidates. Empirically, this leads to sets often exhibiting high overlap, e.g., strings may differ by only a single word. Yet in use-cases that call for multiple solutions, a diverse or representative set is often desired. To address this issue, we propose a reformulation of beam search, which we call determinantal beam search. Determinantal beam search has a natural relationship to determinantal point processes (DPPs), models over sets that inherently encode intra-set interactions. By posing iterations in beam search as a series of subdeterminant maximization problems, we can turn the algorithm into a diverse subset selection process. In a case study, we use the string subsequence kernel to explicitly encourage n-gram coverage in text generated from a sequence model. We observe that our algorithm offers competitive performance against other diverse set generation strategies in the context of language generation, while providing a more general approach to optimizing for diversity.

READ FULL TEXT
09/22/2021

Conditional Poisson Stochastic Beam Search

Beam search is the default decoding strategy for many sequence generatio...
09/17/2019

BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation

This study mainly investigates two decoding problems in neural keyphrase...
10/07/2016

Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

Neural sequence models are widely used to model time-series data in many...
03/08/2022

Beam Search for Feature Selection

In this paper, we present and prove some consistency results about the p...
04/03/2017

Multi-rendezvous Spacecraft Trajectory Optimization with Beam P-ACO

The design of spacecraft trajectories for missions visiting multiple cel...
10/06/2020

If beam search is the answer, what was the question?

Quite surprisingly, exact maximum a posteriori (MAP) decoding of neural ...
09/11/2021

Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy

Generating goal-oriented questions in Visual Dialogue tasks is a challen...