Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems

05/14/2019
by   Fei Mi, et al.
0

Natural language generation (NLG) is an essential component of task-oriented dialogue systems. Despite the recent success of neural approaches for NLG, they are typically developed for particular domains with rich annotated training examples. In this paper, we study NLG in a low-resource setting to generate sentences in new scenarios with handful training examples. We formulate the problem from a meta-learning perspective, and propose a generalized optimization-based approach (Meta-NLG) based on the well-recognized model-agnostic meta-learning (MAML) algorithm. Meta-NLG defines a set of meta tasks, and directly incorporates the objective of adapting to new low-resource NLG tasks into the meta-learning optimization process. Extensive experiments are conducted on a large multi-domain dataset (MultiWoz) with diverse linguistic variations. We show that Meta-NLG significantly outperforms other training procedures in various low-resource configurations. We analyze the results, and demonstrate that Meta-NLG adapts extremely fast and well to low-resource situations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/27/2019

Investigating Meta-Learning Algorithms for Low-Resource Natural Language Understanding Tasks

Learning general representations of text is a fundamental problem for ma...
research
03/22/2022

Improving Meta-learning for Low-resource Text Classification and Generation via Memory Imitation

Building models of natural language processing (NLP) is challenging in l...
research
03/02/2018

Natural Language to Structured Query Generation via Meta-Learning

In conventional supervised training, a model is trained to fit all the t...
research
01/19/2023

A Meta-Learning Approach for Software Refactoring

Software refactoring is the process of changing the structure of softwar...
research
05/11/2022

Improved Meta Learning for Low Resource Speech Recognition

We propose a new meta learning based framework for low resource speech r...
research
03/12/2020

Meta-Learning Initializations for Low-Resource Drug Discovery

Building in silico models to predict chemical properties and activities ...
research
10/04/2020

Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning

Sentence function is an important linguistic feature indicating the comm...

Please sign up or login with your details

Forgot password? Click here to reset