
LatentVariable NonAutoregressive Neural Machine Translation with Deterministic Inference using a Delta Posterior
Although neural machine translation models reached high translation qual...
read it

NonAutoregressive Neural Machine Translation
Existing approaches to neural machine translation condition each output ...
read it

Deterministic NonAutoregressive Neural Sequence Modeling by Iterative Refinement
We propose a conditional nonautoregressive neural sequence model based ...
read it

Imitation Learning for NonAutoregressive Neural Machine Translation
Nonautoregressive translation models (NAT) have achieved impressive inf...
read it

Minimizing the BagofNgrams Difference for NonAutoregressive Neural Machine Translation
NonAutoregressive Neural Machine Translation (NAT) achieves significant...
read it

Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
This paper introduces Dynamic Programming Encoding (DPE), a new segmenta...
read it

Fast Latent Variable Models for Inference and Visualization on Mobile Devices
In this project we outline Vedalia, a high performance distributed netwo...
read it
Iterative Refinement in the Continuous Space for NonAutoregressive Neural Machine Translation
We propose an efficient inference procedure for nonautoregressive machine translation that iteratively refines translation purely in the continuous space. Given a continuous latent variable model for machine translation (Shu et al., 2020), we train an inference network to approximate the gradient of the marginal log probability of the target sentence, using only the latent variable as input. This allows us to use gradientbased optimization to find the target sentence at inference time that approximately maximizes its marginal probability. As each refinement step only involves computation in the latent space of low dimensionality (we use 8 in our experiments), we avoid computational overhead incurred by existing nonautoregressive inference procedures that often refine in token space. We compare our approach to a recently proposed EMlike inference procedure (Shu et al., 2020) that optimizes in a hybrid space, consisting of both discrete and continuous variables. We evaluate our approach on WMT'14 EnDe, WMT'16 RoEn and IWSLT'16 DeEn, and observe two advantages over the EMlike inference: (1) it is computationally efficient, i.e. each refinement step is twice as fast, and (2) it is more effective, resulting in higher marginal probabilities and BLEU scores with the same number of refinement steps. On WMT'14 EnDe, for instance, our approach is able to decode 6.2 times faster than the autoregressive model with minimal degradation to translation quality (0.9 BLEU).
READ FULL TEXT
Comments
There are no comments yet.