Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks

by   Yong Wang, et al.

The key challenge of multi-domain translation lies in simultaneously encoding both the general knowledge shared across domains and the particular knowledge distinctive to each domain in a unified model. Previous work shows that the standard neural machine translation (NMT) model, trained on mixed-domain data, generally captures the general knowledge, but misses the domain-specific knowledge. In response to this problem, we augment NMT model with additional domain transformation networks to transform the general representations to domain-specific representations, which are subsequently fed to the NMT decoder. To guarantee the knowledge transformation, we also propose two complementary supervision signals by leveraging the power of knowledge distillation and adversarial learning. Experimental results on several language pairs, covering both balanced and unbalanced multi-domain translation, demonstrate the effectiveness and universality of the proposed approach. Encouragingly, the proposed unified model achieves comparable results with the fine-tuning approach that requires multiple models to preserve the particular knowledge. Further analyses reveal that the domain transformation networks successfully capture the domain-specific knowledge as expected.


page 1

page 2

page 3

page 4


Iterative Dual Domain Adaptation for Neural Machine Translation

Previous studies on the domain adaptation for neural machine translation...

Multi-Domain Neural Machine Translation

We present an approach to neural machine translation (NMT) that supports...

Mechanisms for producing a working knowledge: Enacting, orchestrating and organizing

Given that knowledge (intensive) work takes place immersed in truly hete...

Finding Sparse Structure for Domain Specific Neural Machine Translation

Fine-tuning is a major approach for domain adaptation in Neural Machine ...

Transformation Importance with Applications to Cosmology

Machine learning lies at the heart of new possibilities for scientific d...

Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing

Many multi-domain neural machine translation (NMT) models achieve knowle...