Determinantal Beam Search

06/14/2021
by   Clara Meister, et al.
0

Beam search is a go-to strategy for decoding neural sequence models. The algorithm can naturally be viewed as a subset optimization problem, albeit one where the corresponding set function does not reflect interactions between candidates. Empirically, this leads to sets often exhibiting high overlap, e.g., strings may differ by only a single word. Yet in use-cases that call for multiple solutions, a diverse or representative set is often desired. To address this issue, we propose a reformulation of beam search, which we call determinantal beam search. Determinantal beam search has a natural relationship to determinantal point processes (DPPs), models over sets that inherently encode intra-set interactions. By posing iterations in beam search as a series of subdeterminant maximization problems, we can turn the algorithm into a diverse subset selection process. In a case study, we use the string subsequence kernel to explicitly encourage n-gram coverage in text generated from a sequence model. We observe that our algorithm offers competitive performance against other diverse set generation strategies in the context of language generation, while providing a more general approach to optimizing for diversity.

READ FULL TEXT
research
09/22/2021

Conditional Poisson Stochastic Beam Search

Beam search is the default decoding strategy for many sequence generatio...
research
09/17/2019

BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation

This study mainly investigates two decoding problems in neural keyphrase...
research
09/07/2023

Conformal Autoregressive Generation: Beam Search with Coverage Guarantees

We introduce two new extensions to the beam search algorithm based on co...
research
10/07/2016

Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

Neural sequence models are widely used to model time-series data in many...
research
03/08/2022

Beam Search for Feature Selection

In this paper, we present and prove some consistency results about the p...
research
04/03/2017

Multi-rendezvous Spacecraft Trajectory Optimization with Beam P-ACO

The design of spacecraft trajectories for missions visiting multiple cel...
research
09/12/2020

Iterative beam search algorithms for the permutation flowshop

We study an iterative beam search algorithm for the permutation flowshop...

Please sign up or login with your details

Forgot password? Click here to reset