Dynamic Multi-Level Multi-Task Learning for Sentence Simplification

06/19/2018
by   Han Guo, et al.
0

Sentence simplification aims to improve readability and understandability, based on several operations such as splitting, deletion, and paraphrasing. However, a valid simplified sentence should also be logically entailed by its input sentence. In this work, we first present a strong pointer-copy mechanism based sequence-to-sequence sentence simplification model, and then improve its entailment and paraphrasing capabilities via multi-task learning with related auxiliary tasks of entailment and paraphrase generation. Moreover, we propose a novel 'multi-level' layered soft sharing approach where each auxiliary task shares different (higher versus lower) level layers of the sentence simplification model, depending on the task's semantic versus lexico-syntactic nature. We also introduce a novel multi-armed bandit based training approach that dynamically learns how to effectively switch across tasks during multi-task learning. Experiments on multiple popular datasets demonstrate that our model outperforms competitive simplification systems in SARI and FKGL automatic metrics, and human evaluation. Further, we present several ablation analyses on alternative layer sharing methods, soft versus hard sharing, dynamic multi-armed bandit sampling approaches, and our model's learned entailment and paraphrasing skills.

READ FULL TEXT
research
05/28/2018

Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation

An accurate abstractive summary of a document should contain all its sal...
research
04/08/2019

AutoSeM: Automatic Task Selection and Mixing in Multi-Task Learning

Multi-task learning (MTL) has achieved success over a wide range of prob...
research
04/21/2018

Multi-task Learning for Universal Sentence Representations: What Syntactic and Semantic Information is Captured?

Learning distributed sentence representations is one of the key challeng...
research
04/24/2017

Multi-Task Video Captioning with Video and Entailment Generation

Video captioning, the task of describing the content of a video, has see...
research
04/12/2016

Improving sentence compression by learning to predict gaze

We show how eye-tracking corpora can be used to improve sentence compres...
research
06/17/2022

Thompson Sampling for Robust Transfer in Multi-Task Bandits

We study the problem of online multi-task learning where the tasks are p...
research
05/10/2023

Efficient Training of Multi-task Neural Solver with Multi-armed Bandits

Efficiently training a multi-task neural solver for various combinatoria...

Please sign up or login with your details

Forgot password? Click here to reset