DeepAI AI Chat
Log In Sign Up

Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework

by   Shaolei Zhang, et al.

Simultaneous machine translation (SiMT) starts translating while receiving the streaming source inputs, and hence the source sentence is always incomplete during translating. Different from the full-sentence MT using the conventional seq-to-seq architecture, SiMT often applies prefix-to-prefix architecture, which forces each target word to only align with a partial source prefix to adapt to the incomplete source in streaming inputs. However, the source words in the front positions are always illusoryly considered more important since they appear in more prefixes, resulting in position bias, which makes the model pay more attention on the front source positions in testing. In this paper, we first analyze the phenomenon of position bias in SiMT, and develop a Length-Aware Framework to reduce the position bias by bridging the structural gap between SiMT and full-sentence MT. Specifically, given the streaming inputs, we first predict the full-sentence length and then fill the future source position with positional encoding, thereby turning the streaming inputs into a pseudo full-sentence. The proposed framework can be integrated into most existing SiMT methods to further improve performance. Experiments on two representative SiMT methods, including the state-of-the-art adaptive policy, show that our method successfully reduces the position bias and thereby achieves better SiMT performance.


From Simultaneous to Streaming Machine Translation by Leveraging Streaming History

Simultaneous Machine Translation is the task of incrementally translatin...

Gaussian Multi-head Attention for Simultaneous Machine Translation

Simultaneous machine translation (SiMT) outputs translation while receiv...

Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation

Simultaneous machine translation (SiMT) starts its translation before re...

Transformer-based Automatic Post-Editing with a Context-Aware Encoding Approach for Multi-Source Inputs

Recent approaches to the Automatic Post-Editing (APE) research have show...

Re-translation versus Streaming for Simultaneous Translation

There has been great progress in improving streaming machine translation...

Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy

Simultaneous translation, which starts translating each sentence after r...

A Continuum of Generation Tasks for Investigating Length Bias and Degenerate Repetition

Language models suffer from various degenerate behaviors. These differ b...