DeepAI AI Chat
Log In Sign Up

Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings

08/27/2019
by   Zi-Yi Dou, et al.
Carnegie Mellon University
0

The recent success of neural machine translation models relies on the availability of high quality, in-domain data. Domain adaptation is required when domain-specific data is scarce or nonexistent. Previous unsupervised domain adaptation strategies include training the model with in-domain copied monolingual or back-translated data. However, these methods use generic representations for text regardless of domain shift, which makes it infeasible for translation models to control outputs conditional on a specific domain. In this work, we propose an approach that adapts models with domain-aware feature embeddings, which are learned via an auxiliary language modeling task. Our approach allows the model to assign domain-specific representations to words and output sentences in the desired domain. Our empirical results demonstrate the effectiveness of the proposed strategy, achieving consistent improvements in multiple experimental settings. In addition, we show that combining our method with back translation can further improve the performance of the model.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/26/2019

An Empirical Study of Domain Adaptation for Unsupervised Neural Machine Translation

Domain adaptation methods have been well-studied in supervised neural ma...
10/07/2019

Domain Differential Adaptation for Neural Machine Translation

Neural networks are known to be data hungry and domain sensitive, but it...
09/14/2021

Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation

Recently, kNN-MT has shown the promising capability of directly incorpor...
12/11/2021

Selecting Parallel In-domain Sentences for Neural Machine Translation Using Monolingual Texts

Continuously-growing data volumes lead to larger generic models. Specifi...
11/05/2018

Compact Personalized Models for Neural Machine Translation

We propose and compare methods for gradient-based domain adaptation of s...
04/08/2019

Improving Domain Adaptation Translation with Domain Invariant and Specific Information

In domain adaptation for neural machine translation, translation perform...
04/05/2020

Unsupervised Domain Clusters in Pretrained Language Models

The notion of "in-domain data" in NLP is often over-simplistic and vague...