Language Scaling for Universal Suggested Replies Model

by   Qianlan Ying, et al.

We consider the problem of scaling automated suggested replies for Outlook email system to multiple languages. Faced with increased compute requirements and low resources for language expansion, we build a single universal model for improving the quality and reducing run-time costs of our production system. However, restricted data movement across regional centers prevents joint training across languages. To this end, we propose a multi-task continual learning framework, with auxiliary tasks and language adapters to learn universal language representation across regions. The experimental results show positive cross-lingual transfer across languages while reducing catastrophic forgetting across regions. Our online results on real user traffic show significant gains in CTR and characters saved, as well as 65 reduction compared with per-language models. As a consequence, we have scaled the feature in multiple languages including low-resource markets.


page 1

page 2

page 3

page 4


Unsupervised Cross-lingual Representation Learning at Scale

This paper shows that pretraining multilingual language models at scale ...

ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation

Despite the recent advancement in NLP research, cross-lingual transfer f...

Cross-lingual Lifelong Learning

The longstanding goal of multi-lingual learning has been to develop a un...

On Learning Universal Representations Across Languages

Recent studies have demonstrated the overwhelming advantage of cross-lin...

Cross-lingual Transfer Learning for Fake News Detector in a Low-Resource Language

Development of methods to detect fake news (FN) in low-resource language...

Learning to pronounce as measuring cross-lingual joint orthography-phonology complexity

Machine learning models allow us to compare languages by showing how har...