Neural sequence generation models are widely developed and deployed in tasks such as machine translation (Bahdanau et al., 2015; Vaswani et al., 2017). As we examine the current frameworks, the most popular autoregressive models generate tokens step-by-step. If not better, recent non-autoregressive approaches (Gu et al., 2018; Kaiser et al., 2018; Lee et al., 2018) have proved it possible to perform generation within a much smaller number of decoding iterations.
In this paper, we propose Levenshtein Transformer (LevT), aiming to address the lack of flexibility of the current decoding models. Notably, in the existing frameworks, the length of generated sequences is either fixed or monotonically increased as the decoding proceeds. This remains incompatible with human-level intelligence where humans can revise, replace, revoke or delete any part of their generated text. Hence, LevT is proposed to bridge this gap by breaking the in-so-far standardized decoding mechanism and replacing it with two atomic operations —insertion and deletion.
We train the LevT using imitation learning. The resulted model contains two policies and they are executed in an alternate manner. Empirically, we show that LevT achieves comparable or better results than a standard Transformer model on machine translation and summarization, while maintaining the efficiency advantages benefited from parallel decoding similarly to(Lee et al., 2018). With this model, we argue that the decoding becomes more flexible. For example, when the decoder is given an empty token, it falls back to a normal sequence generation model. On the other hand, the decoder acts as a refinement model when the initial state is a low-quality generated sequence. Indeed, we show that a LevT trained from machine translation is directly applicable to translation post-editing without any change. This would not be possible with any framework in the literature because generation and refinement are treated as two different tasks due to the model’s inductive bias.
One crucial component in LevT framework is the learning algorithm. We leverage the characteristics of insertion and deletion — they are complementary but also adversarial. The algorithm we propose is called “dual policy learning”. The idea is that when training one policy (insertion or deletion), we use the output from its adversary at the previous iteration as input. An expert policy, on the other hand, is drawn to provide a correction signal. Despite that, in theory, this learning algorithm is applicable to other imitation learning scenarios where a dual adversarial policy exists, in this work we primarily focus on a proof-of-concept of this algorithm landing at training the proposed LevT model.
To this end, we summarize the contributions as follows:
We propose Levenshtein Transformer (LevT), a new sequence generation model composed of the insertion and deletion operations. This model achieves comparable or better results than a strong Transformer baseline in both machine translation and text summarization, but with much better efficiency (up to speed-up);
We propose a corresponding learning algorithm under the theoretical framework of imitation learning, tackling the complementary and adversarial nature of the dual policies;
We recognize our model as a pioneer attempt to unify sequence generation and refinement, thanks to its built-in flexibility. With this unification, we empirically validate the feasibility of applying a LevT model trained by machine translation directly to translation post-editing, without any change.
2 Problem Formulation
2.1 Sequence Generation and Refinement
We unify the general problems of sequence generation and refinement by casting them to a Markov Decision Process (MDP) defined by a tuple. We consider the setup consisting an agent interacting with an environment which receives the agent’s editing actions and returns the modified sequence. We define as a set of discrete sequences up to length where is a vocabulary of symbols. At every decoding iteration, the agent receives an input drawn from scratch or uncompleted generation, chooses an action and gets a reward . We use to denote the set of actions and for the reward function. Generally the reward function measures the distance between the generation and the ground-truth sequence, which can be any distance measurement such as the Levenshtein distance (Levenshtein, 1965). It is crucial to incorporate into the our formulation. As the initial sequence, the agent receives—when is an already generated sequence from another system, the agent essentially learns to do refinement while it falls back to generation if is an empty sequence. The agent is modeled by a policy,
, that maps the current generation over a probability distribution over. That is, .
2.2 Actions: Deletion & Insertion
Following the above MDP formulation, with a subsequence , the two atomic actions – deletion and insertion – are called to generate . Here we let and be special symbols <s> and </s>, respectively. Since we mainly focus on the policy of a single round generation, the superscripts are normally omitted in this section. For conditional generation like MT, our policy also includes an input of source information which is also omitted here.
The deletion policy reads the input sequence , and for every token , the deletion policy makes a binary decision which is 1 (delete this token) or 0 (keep it). We additionally constrain to avoid sequence boundary being broken.
It is slightly more complex to build the insertion atomic because it involves two phases: placeholder prediction and token prediction so that it is able to insert multiple tokens at the same slot. First, among all the possible inserted slots () in , predicts the possibility of adding one or several placeholders. In what follows, for every placeholder predicted as above, a token prediction policy replaces the placeholders with actual tokens in the vocabulary.
Recall that our two atomic operations are complementary. Hence we combine them in an alternate fashion. For example in sequence generation from the empty, insertion policy is first called and it is followed by deletion, and then repeat till the certain stopping condition is fulfilled. Indeed, it is possible to leverage the parallelism in this combination. We essentially decompose one iteration of our sequence generator into three phases: “delete tokens – insert placeholders – replace placeholders with new tokens”. Within each stage, all operations are performed in parallel. More precisely, given the current sequence , and suppose the action to predict is , we our policy for one iteration is:
where and . We parallelize the computation within each sub-tasks.
3 Levenshtein Transformer
In this section, we cover the specs of Levenshtein Transformer and the dual-policy learning algorithm. Overall our model takes a sequence of tokens (or none) as the input then iteratively modify it by alternating between insertion and deletion, until the two policies combined converge. We describe the detailed learning and inference algorithms in the Appendix.
We use Transformer (Vaswani et al., 2017) as the basic building block. For conditional generation, the source is included in each TransformerBlock. The states from the -th block are:
where and are the token and position embeddings, respectively.
The decoder outputs () are passed to three policy classifiers:
Deletion Classifier: LevT scans over the input tokens (except for the boundaries) and predict “deleted” () or “kept” () for each token position,
where , and we always keep the boundary tokens.
Placeholder Classifier: LevT predicts the number of tokens to be inserted at every consecutive position pairs, by casting the representation to a categorical distribution:
where . Based on the number () of tokens it predicts, we insert the considered number of placeholders at the current position. In our implementation, placehoder is represented by a special token <PLH> which was reserved in the vocabulary.
Token Classifier: following the placeholder prediction, LevT needs to fill in tokens replacing all the placeholders. This is achieved by training a token predictor as follow:
where with parameters being shared with the embedding matrix.
Although it is parameter-efficient to share the same Transformer architecture across the above three heads, there is room for improvement as one decoding iteration requires three full passes of the network. To make trade-off between performance and computational cost, we propose to perform early exit (attaching the head to an intermediate block instead of the last one) for and while keeping always based on the last block, considering that token prediction is usually more challenging than the other two tasks.
3.2 Dual-policy Learning
We use imitation learning to train the Levenshtein Transformer. Essentially we let the agent imitate the behaviors that we draw from some expert policy . The expert policy is derived from direct usage of ground-truth targets or less noisy version filtered by sequence distillation (Kim and Rush, 2016). The objective is to maximize the following expectation:
where is the output after inserting palceholders upon . , are the roll-in polices and we repeatedly draw states (sequences) from their induced state distribution . These states are first executed by the expert policy returning the suggested actions by the expert, and then we maximize the conditional log-likelihood over them.
By definition, the roll-in policy determines the state distribution fed to during training. In this work, we have two strategies to construct the roll-in policy — adding noise to the ground-truth or using the output from the adversary policy. Figure 2 shows a diagram of this learning paradigm. We formally write down the roll-in policies as follows.
Learning to Delete: we design the as a stochastic mixture between the initial input or the output by applying insertion from the model with some mixture factor :
where and is any sequence ready to insert tokens. is obtained by sampling instead of doing argmax from Eq. (5).
Learning to Insert: similar to the deletion step, we apply a mixture of the deletion output and a random word dropping sequence of the round-truth, inspired by recent advances of training masked language model (Devlin et al., 2018). We use random dropping as a form of noise injection to encourage more exploration. Let and ,
It is crucial to construct an expert policy in imitation learning which cannot be too hard or too weak to learn from. Specifically, we considered two types of experts:
Oracle: One way is to build an oracle which accesses to the ground-truth sequence. It returns the optimal actions (either oracle insertion or oracle deletion ) by:
Here, we use the Levenshtein distance (Levenshtein, 1965)111We only consider the variant which only computes insertion and deletion. No substitution is considered. as considering it is possible to obtain the action suggestions efficiently by dynamic programming.
Teacher Model: We also explore to use another teacher model to provide expert policy, which is known as knowledge distillation (Kim and Rush, 2016). This technique has been widely used in previous approaches for nonauoregressive generation (Gu et al., 2018). More precisely, we first train an autoregressive teacher model using the same datasets and then replace the ground-truth sequence by the beam-search result of this teacher-model, . We use the same mechanism to find the suggested option as using the ground-truth oracle.
At inference time, we apply the trained model over the initial sequence for several iterations. We greedily pick up the actions associated with high probabilities in Eq. (3)(4)(5). Moreover, we find that using search (instead of greedy decoding) does not yield much gain in LevT. This observation is quite opposite to what has been widely discovered in autoregressive decoding. We hypothesize this is because the local optimal point brought by greedy decoding in autoregressive models is often far from the optimality point globally. Search techniques resolve this issue with tabularization. In our case, however, because LevT inserts or deletes tokens dynamically, it could easily revoke the tokens that are found sub-optimal and re-insert better ones.
The decoding stops when one of the following conditions is fulfilled:
Nothing to delete, Nothing to insert: The policy chooses to keep all the current tokens, and predicts “empty” placeholders at everywhere.
Direct-Loop: Unfortunately, our MDP assumption cannot avoid the situations where the agent gets stuck in an infinite loop; i.e. the insertion and deletion counter each other and keep looping. Although this is not common, we terminate the decoding once this is spotted.
Penalty for Empty Placeholders
We validate the efficiency, effectiveness, and flexibility of Levenshtein Transformer extensively across three different tasks — machine translation (MT), text summarization (TS) and automatic post-editing (APE) for machine translation, from both generation (§4.1) and refinement (§4.2) perspectives.
4.1 Sequence Generation
For the sequence generation perspective, we evaluate LevT model on MT and TS. As a special case, sequence generation assumes empty as input and no initial deletion is applied.
Data & Evaluation
We use three diversified language pairs for MT experiments: WMT’16 Romanian-English (Ro-En)222http://www.statmt.org/wmt16/translation-task.html, WMT’14 English-German (En-De)333http://www.statmt.org/wmt14/translation-task.html and WAT2017 Small-NMT English-Japanese (En-Ja)444http://lotus.kuee.kyoto-u.ac.jp/WAT/WAT2017/snmt/index.html. The TS experiments use preprocessed data from the Annotated English Gigaword (Rush et al., 2015)555https://github.com/harvardnlp/sent-summary. We learn byte-pair encoding (BPE, Sennrich et al., 2016)
vocabulary on tokenized data. Detailed dataset statistics can be found in the Appendix. For evaluation metrics, we use BLEU(Papineni et al., 2002) for MT and ROUGE-1,2,L (Lin, 2004) for TS. Before computing the BLEU scores for Japanese output, we always segment Japanese words using KyTea 666http://www.phontron.com/kytea/.
Models & Training
We adopt the model architecture of Transformer base (Vaswani et al., 2017) for the proposed LevT model and the autoregressive baseline. All the Transformer-based models are trained on Nvidia Volta GPUs with maximum steps and a total batch-size of around tokens per step (We leave more details to the Appendix).
|Speed||Ro-En||Latency (ms) /||326 / 27.1||349 / 27.1||97 / 2.19|
|En-De||Latency (ms) /||343 / 28.1||369 / 28.1||126 / 2.88||92 / 2.05|
|En-Ja||Latency (ms) /||261 / 22.6||306 / 22.6||112 / 2.61||106 / 1.97|
|Gigaword||Latency (ms) /||116 / 10.1||149 / 10.1||98 / 2.32||84 / 1.73|
We present our main results on the generation quality and decoding speed in Table 1. We measure the speed by the averaged generation latency of generating one sequence at a time on single Nvidia V100 GPU. To remove the implementation bias, we also present the number of decoder iterations as a reference. It can be concluded that for both MT and summarization tasks, our proposed LevT achieves comparable and sometimes better generation quality compared to the strong autoregressive baseline, while LevT is much more efficient at decoding. A translation example is shown in Figure 3 and we leave more in Appendix.
Oracle v.s. Teacher Model
As shown in Table 1, training the LevT with the teacher model achieves better results than the oracle counterpart, in most of the cases. Another interesting observation is the model has smaller latency. We conjecture that this is due to that the output of the teacher model possesses fewer modes and much less noisy than the real data. Consequently, LevT needs less number of iterations to converge to this expert policy.
Analysis of Efficiency
As shown in Figure 4 (a), our model learns to properly terminate the decoding and adjust it based on the length of input. We also explore the variants of “early exit” where we denote LevT(-) as a model with and blocks for deletion (Eq. (3)) and placeholder prediction (Eq. (4)) respectively. Figure 4 (b) shows that althought it compromises the generation quality a bit, our model with early exit achieves up to speed-up with on-par performance comparing against a strong autoregressive Transformer using beam-search.
Importance of mixture roll-in policy
We perform an ablation study on the learning algorithm. Specifically, we train a model with no mixing of the in Equation (6). We name this experiment by
due to its resemblance to a denoising autoencoder. We follow closely a standard pipeline established byLee et al. (2018). Table 2 shows this comparison. As we can see that the deletion loss from is much smaller while the generation BLEU score is inferior. We conjecture that this is caused by the mismatch between the states from the model and the roll-in policy in training the .
4.2 Sequence Refinement
We evaluate LevT’s capability of refining sequence outputs on the APE task. In this setting, inputs are pairs of the source sequence and a black-box MT system generation. The ground-truth outputs are from real human edits with expansion using synthetic data.
We follow a normal protocol in the synthetic APE experiments (Grangier and Auli, 2017): we first train the input MT system on half of the dataset. Then we will train a refinement model on the other half based on the output produced by the MT model trained in the previous phase. For the real APE tasks, we use the data from WMT17 Automatic Post-Editing Shared Task999http://www.statmt.org/wmt17/ape-task.html on En-De. It contains both real PE triples and a large-scale synthetic corpus.
Models & Evaluation
The baseline model is a standard Transformer encoding the concatenation of the source and the MT system’s output. For the MT system here, we want some imperfect systems that need to be refined. We consider a statistical phrase-based MT system (PBMT, Koehn et al., 2003) and an RNN-based NMT system (Bahdanau et al., 2015). Apart from BLEU scores, we additionally apply translation error rate (TER, Snover et al., 2006) as it is widely used in the APE literature.
|Synthetic||Ro-En||PBMT||27.5 / 52.6||28.9 / 52.8||29.1 / 50.4||30.1 / 51.7|
|NMT||26.2 / 56.5||26.9 / 55.6||28.3 / 53.6||28.0 / 55.8|
|En-De||PBMT||15.4 / 69.4||22.8 / 61.0||25.8 / 56.6||16.5 / 69.6|
|En-Ja||NMT||37.7 / 48.0||41.0 / 44.9||42.2 / 44.3||39.4 / 47.5|
|Real||En-De||PBMT||62.5 / 24.5||67.2 / 22.1||66.9 / 21.9||59.6 / 28.7||70.1 / 19.2|
We show the major comparison in Table 3. When training from scratch, LevT consistently improves the performance of the input MT system (either PBMT or NMT). It also achieves better performance than the autoregressive Transformer in most of the cases.
Pre-training on MT
Thanks to the generality of the LevT model, we show it is feasible to directly apply the LevT model trained by generation onto refinement tasks — in this case — MT and APE. We name this a “zero-shot post-editing” setting. According to Table 3, the pre-trained MT models are always capable of improving the initial MT input in the synthetic tasks.
The real APE task, however, differs quite a bit from the synthetic tasks because human translators normally only fix a few spotted errors. This ends up with very high BLEU scores even for the “Do-nothing” column. However, the pre-trained MT model achieves the best results by fine-tuning on the PE data indicating that LevT is able to leverage the knowledge for generation and refinement.
Collaborate with Oracle
Thanks to the saperation of insertion and deletion operations, LevT has better interpretability and controllability. For example, we test the ability that LevT adapts oracle (e.g. human translators) instructions. As shown in Figure 5, both MT and PE tasks have huge improvement if every step the oracle deletion is given. This goes even further if the oracle provides both the correct deletion and the number of placehoders to insert. It also sheds some light upon computer-assisted text editing for human translators.
5 Related Work
Non-Autoregressive or Non-Monotonic Decoding
Breaking the autoregressive constraints and monotonic (left-to-right) decoding order in classic neural sequence generation systems has recently attracted much interest. Stern et al. (2018); Wang et al. (2018) designed partially parallel decoding schemes to output multiple tokens at each step. Gu et al. (2018) proposed a non-autoregressive framework using discrete latent variables, which was later adopted in Lee et al. (2018) as iterative refinement process. Ghazvininejad et al. (2019) introduced the masked language modeling objective from BERT (Devlin et al., 2018) to non-autoregressively predict and refine translations. Welleck et al. (2019); Stern et al. (2019); Gu et al. (2019) generate translations non-monotonically by adding words to the left or right of previous ones or by inserting words in arbitrary order to form a sequence.
Novak et al. (2016)
predict and apply token substitutions iteratively on phase-based MT system outputs using convolutional neural network. QuickEdit(Grangier and Auli, 2017) and deliberation network (Xia et al., 2017) both consist of two autoregressive decoders where the second decoder refines the translation generated by the first decoder. Guu et al. (2018) propose a neural editor which learned language modeling by first retrieving a prototype and then editing over that. Freitag et al. (2019) correct patterned errors in MT system outputs using transformer models trained on monolingual data.
We propose Levenshtein Transformer, a neural sequence generation model based on insertion and deletion. The resulted model achieves performance and decoding efficiency, and embraces sequence generation to refinement in one model. The insertion and deletion operations are arguably more similar to how human writes or edits text. For future work, it is potential to extend this model to human-in-the-loop generation.
We would like to thank Kyunghyun Cho, Marc’Aurelio Ranzato, Douwe Kiela, Qi Liu and our colleagues at Facebook AI Research for valuable feedback, discussions and technical assistance.
- Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
- Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
- Freitag et al. (2019) Markus Freitag, Isaac Caswell, and Scott Roy. 2019. Text repair model for neural machine translation. arXiv preprint arXiv:1904.04790.
- Ghazvininejad et al. (2019) Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. Constant-time machine translation with conditional masked language models. CoRR, abs/1904.09324.
- Grangier and Auli (2017) David Grangier and Michael Auli. 2017. Quickedit: Editing text & translations by crossing words out. arXiv preprint arXiv:1711.04805.
Gu et al. (2018)
Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher.
Non-autoregressive neural machine translation.In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, Canada, April 30-May 3, 2018, Conference Track Proceedings.
- Gu et al. (2019) Jiatao Gu, Qi Liu, and Kyunghyun Cho. 2019. Insertion-based decoding with automatically inferred generation order. arXiv preprint arXiv:1902.01370.
- Guu et al. (2018) Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. 2018. Generating sentences by editing prototypes. Transactions of the Association of Computational Linguistics, 6:437–450.
Kaiser et al. (2018)
Lukasz Kaiser, Samy Bengio, Aurko Roy, Ashish Vaswani, Niki Parmar, Jakob
Uszkoreit, and Noam Shazeer. 2018.
Fast decoding in sequence models using discrete latent variables.
International Conference on Machine Learning, pages 2395–2404.
- Kim and Rush (2016) Yoon Kim and Alexander Rush. 2016. Sequence-level knowledge distillation. In EMNLP.
- Koehn et al. (2003) Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 48–54. Association for Computational Linguistics.
Lee et al. (2018)
Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018.
Deterministic non-autoregressive neural sequence modeling by iterative
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 1173–1182.
- Levenshtein (1965) Vladimir Iosifovich Levenshtein. 1965. Binary codes capable of correcting deletions, insertions, and reversals. In Doklady Akademii Nauk, volume 163, pages 845–848. Russian Academy of Sciences.
- Lin (2004) Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
- Novak et al. (2016) Roman Novak, Michael Auli, and David Grangier. 2016. Iterative refinement for machine translation. arXiv preprint arXiv:1610.06602.
- Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
- Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379–389, Lisbon, Portugal. Association for Computational Linguistics.
- Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
- Snover et al. (2006) Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In In Proceedings of Association for Machine Translation in the Americas, pages 223–231.
- Stern et al. (2019) Mitchell Stern, William Chan, Jamie Kiros, and Jakob Uszkoreit. 2019. Insertion transformer: Flexible sequence generation via insertion operations. arXiv preprint arXiv:1902.03249.
- Stern et al. (2018) Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit. 2018. Blockwise parallel decoding for deep autoregressive models. In Advances in Neural Information Processing Systems, pages 10107–10116.
- Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS).
- Wang et al. (2018) Chunqi Wang, Ji Zhang, and Haiqing Chen. 2018. Semi-autoregressive neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 479–488, Brussels, Belgium. Association for Computational Linguistics.
- Welleck et al. (2019) Sean Welleck, Kianté Brantley, Hal Daumé III, and Kyunghyun Cho. 2019. Non-monotonic sequential text generation. arXiv preprint arXiv:1902.02192.
- Xia et al. (2017) Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2017. Deliberation networks: Sequence generation beyond one-pass decoding. In Advances in Neural Information Processing Systems, pages 1784–1794.
Appendix A Learning & Inference Algorithm
We present the detailed algorithms for learning and decoding from Levenshtein Transformer as follows. For simplicity, we always omit the source information in conditional sequence generation tasks such as machine translation which is handled by the cross-attention with an encoder on .
The learning algorithm is shown in Algorithm 1. is the environment and is denoted as the Levenshtein distance, and we can easily back-track the optimal insertion and deletion operations through dynamic programming. We only show the the case with single batch-size for convenience. We also present the inference algorithm in Algorithm 2. If the initial sequence is empty (<s></s>), the proposed model will skip the first deletion and do sequence generation. Otherwise, the model starts with deletion operations and refine the input sequence.
Appendix B Dataset and Preprocessing Details
Table 4 and 5 list the statistics ( of sentences, vocabulary) for all the datasets used in this work. We learn BPE vocabulary with joint operations for WMT En-De and Gigaword and joint operations for WMT Ro-En. For WAT En-Ja, we adopt the official BPE vocabularies learned separately on source and target side.
|WAT’17 En-Ja||2,000,000||1790||1812||17,952 / 17,801|
|WAT’17 En-Ja||1,000,000||1,000,000||1790||1812||17,952 / 17,801|
Appendix C Model and Training Details
c.1 Sequence Generation Tasks
Transformer models are used for autoregressive baselines as well as teacher models (for the expert policy). By default, we set , , , , , , and . Source and target side share embeddings in all the training pairs except for WAT En-Ja where BPE vocabularies of both side are learned separately and are almost non-overlapping.
c.2 Sequence Refinement Tasks
For synthetic APE tasks, we keep the same training conditions for LevT as those for MT tasks (§C.1). As described earlier in §4.2, we build the baseline Transformer by concatenating the source and MT system’s output as the input sequence for the encoder. Specially, we restart the positional embeddings for the MT output, add an additional language embedding for each token of the input sequence to show its language type. The detailed hyperpameters are the same as the standard Transformer.
As described in §4.2, we consider the following two different imperfect MT systems to provide the refinement inputs. Firstly, we consider the traditional statistical phrase-based machine translation system (PBMT). We follow the instruction to build the basic baseline model via moses101010http://www.statmt.org/moses/?n=Moses.Baseline. As for the NMT-based model, we use a single layer attention-based model composed by LSTM. We build this model on fairseq-py111111https://github.com/pytorch/fairseq/blob/master/fairseq/models/lstm.py with the default configuration.
For the real APE task, we follow the procedures introduced in Junczys-Dowmunt and Grundkiewicz (2016). Synthetic corpus has two subsets: a 500K one and a 4M one. We over-sample real data by 10 times and merge it with the 500K synthetic data to train APE models. Besides, we also train a LevT MT model on the bigger (4M) synthetic corpus where we only use the source and target pairs.
Appendix D Balanced Speed Test on All Lengths
We see from figure 4 (a) that most of the translations are gotten within iterations. Long sentences (e.g. with over 100 tokens) however are relatively underrepresented in the validation set and have sparser data points than that for shorter ones.
To mitigate this bias, we try adding extra data points for Ro-En by selecting and translating long sentences from a monolingual corpus based on New Crawl. Specifically, we group Ro sentences based on their lengths from and for each group, we randomly sample sentences to decode for each group. We show the averaged number of decoding iterations of each length group in Figure 6.
We see that the time-complexity of the proposed LevT is approximately linear (not constant) to the input length (), but with a much smaller ratio ( iteration tokens) compared to the standard auto-regressive modes ( iteration token).
Appendix E More Decoding Examples
We present more examples from the proposed Levenshtein Transformer as follows.