Deep Transfer Reinforcement Learning for Text Summarization

10/15/2018
by   Yaser Keneshloo, et al.
6

Deep neural networks are data hungry models and thus they face difficulties when used for training on small size data. Transfer learning is a method that could potentially help in such situations. Although transfer learning achieved great success in image processing, its effect in the text domain is yet to be well established especially due to several intricacies that arise in the context of document analysis and understanding. In this paper, we study the problem of transfer learning for text summarization and discuss why the existing state-of-the-art models for this problem fail to generalize well on other (unseen) datasets. We propose a reinforcement learning framework based on self-critic policy gradient method which solves this problem and achieves good generalization and state-of-the-art results on a variety of datasets. Through an extensive set of experiments, we also show the ability of our proposed framework in fine-tuning the text summarization model only with a few training samples. To the best of our knowledge, this is first work that studies transfer learning in text summarization and provides a generic solution that works well on unseen data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/16/2020

Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement

Fine-tuning deep neural networks pre-trained on large scale datasets is ...
research
07/10/2020

Sample-based Regularization: A Transfer Learning Strategy Toward Better Generalization

Training a deep neural network with a small amount of data is a challeng...
research
10/23/2019

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Transfer learning, where a model is first pre-trained on a data-rich tas...
research
05/23/2017

Better Text Understanding Through Image-To-Text Transfer

Generic text embeddings are successfully used in a variety of tasks. How...
research
11/27/2019

Transfer Learning in Visual and Relational Reasoning

Transfer learning is becoming the de facto solution for vision and text ...
research
05/31/2018

Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Deep Reinforcement Learning has managed to achieve state-of-the-art resu...
research
09/30/2019

A Closer Look at Data Bias in Neural Extractive Summarization Models

In this paper, we take stock of the current state of summarization datas...

Please sign up or login with your details

Forgot password? Click here to reset