What Do You Get When You Cross Beam Search with Nucleus Sampling?

07/20/2021
by   Uri Shaham, et al.
0

We combine beam search with the probabilistic pruning technique of nucleus sampling to create two deterministic nucleus search algorithms for natural language generation. The first algorithm, p-exact search, locally prunes the next-token distribution and performs an exact search over the remaining space. The second algorithm, dynamic beam search, shrinks and expands the beam size according to the entropy of the candidate's probability distribution. Despite the probabilistic intuition behind nucleus search, experiments on machine translation and summarization benchmarks show that both algorithms reach the same performance levels as standard beam search.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/25/2023

On the Depth between Beam Search and Exhaustive Search for Text Generation

Beam search and exhaustive search are two extreme ends of text decoding ...
research
04/03/2017

Multi-rendezvous Spacecraft Trajectory Optimization with Beam P-ACO

The design of spacecraft trajectories for missions visiting multiple cel...
research
06/27/2020

Learning Optimal Tree Models Under Beam Search

Retrieving relevant targets from an extremely large target set under com...
research
12/06/2022

Improved Beam Search for Hallucination Mitigation in Abstractive Summarization

Advancement in large pretrained language models has significantly improv...
research
08/23/2005

Search Process and Probabilistic Bifix Approach

An analytical approach to a search process is a mathematical prerequisit...
research
09/12/2019

Speculative Beam Search for Simultaneous Translation

Beam search is universally used in full-sentence translation but its app...
research
10/06/2020

If beam search is the answer, what was the question?

Quite surprisingly, exact maximum a posteriori (MAP) decoding of neural ...

Please sign up or login with your details

Forgot password? Click here to reset