Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

08/28/2018
by   Yilin Yang, et al.
1

Beam search is widely used in neural machine translation, and usually improves translation quality compared to greedy search. It has been widely observed that, however, beam sizes larger than 5 hurt translation quality. We explain why this happens, and propose several methods to address this problem. Furthermore, we discuss the optimal stopping criteria for these methods. Results show that our hyperparameter-free methods outperform the widely-used hyperparameter-free heuristic of length normalization by +2.0 BLEU, and achieve the best results among all methods on Chinese-to-English translation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2019

Learning to Stop in Structured Prediction for Neural Machine Translation

Beam search optimization resolves many issues in neural machine translat...
research
08/31/2018

When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)

In neural text generation such as neural machine translation, summarizat...
research
06/22/2017

Neural Machine Translation with Gumbel-Greedy Decoding

Previous neural machine translation models used some heuristic search al...
research
06/29/2021

Rethinking the Evaluation of Neural Machine Translation

The evaluation of neural machine translation systems is usually built up...
research
04/21/2018

A Stable and Effective Learning Strategy for Trainable Greedy Decoding

As a widely used approximate search strategy for neural network decoders...
research
06/10/2021

Input Augmentation Improves Constrained Beam Search for Neural Machine Translation: NTT at WAT 2021

This paper describes our systems that were submitted to the restricted t...
research
12/16/2021

Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modeling

Neural autoregressive sequence models smear the probability among many p...

Please sign up or login with your details

Forgot password? Click here to reset