Modelling Latent Skills for Multitask Language Generation

02/21/2020
by   Kris Cao, et al.
0

We present a generative model for multitask conditional language generation. Our guiding hypothesis is that a shared set of latent skills underlies many disparate language generation tasks, and that explicitly modelling these skills in a task embedding space can help with both positive transfer across tasks and with efficient adaptation to new tasks. We instantiate this task embedding space as a latent variable in a latent variable sequence-to-sequence model. We evaluate this hypothesis by curating a series of monolingual text-to-text language generation datasets - covering a broad range of tasks and domains - and comparing the performance of models both in the multitask and few-shot regimes. We show that our latent task variable model outperforms other sequence-to-sequence baselines on average across tasks in the multitask setting. In the few-shot learning setting on an unseen test dataset (i.e., a new task), we demonstrate that model adaptation based on inference in the latent task space is more robust than standard fine-tuning based parameter adaptation and performs comparably in terms of overall performance. Finally, we examine the latent task representations learnt by our model and show that they cluster tasks in a natural way.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/28/2022

Combining Modular Skills in Multitask Learning

A modular design encourages neural models to disentangle and recombine d...
research
02/19/2023

Few-shot Multimodal Multitask Multilingual Learning

While few-shot learning as a transfer learning paradigm has gained signi...
research
02/17/2020

On the Discrepancy between Density Estimation and Sequence Generation

Many sequence-to-sequence generation tasks, including machine translatio...
research
11/11/2019

Emotional Voice Conversion using multitask learning with Text-to-speech

Voice conversion (VC) is a task to transform a person's voice to differe...
research
08/09/2014

Bayesian Multitask Learning with Latent Hierarchies

We learn multiple hypotheses for related tasks under a latent hierarchic...
research
06/28/2023

SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills

Traditional multitask learning methods basically can only exploit common...
research
02/13/2017

Multitask diffusion adaptation over networks with common latent representations

Online learning with streaming data in a distributed and collaborative m...

Please sign up or login with your details

Forgot password? Click here to reset