DeepAI AI Chat
Log In Sign Up

Learning to Selectively Transfer: Reinforced Transfer Learning for Deep Text Matching

by   Chen Qu, et al.
University of Massachusetts Amherst
Alibaba Group
Carnegie Mellon University

Deep text matching approaches have been widely studied for many applications including question answering and information retrieval systems. To deal with a domain that has insufficient labeled data, these approaches can be used in a Transfer Learning (TL) setting to leverage labeled data from a resource-rich source domain. To achieve better performance, source domain data selection is essential in this process to prevent the "negative transfer" problem. However, the emerging deep transfer models do not fit well with most existing data selection methods, because the data selection policy and the transfer learning model are not jointly trained, leading to sub-optimal training efficiency. In this paper, we propose a novel reinforced data selector to select high-quality source domain data to help the TL model. Specifically, the data selector "acts" on the source domain data to find a subset for optimization of the TL model, and the performance of the TL model can provide "rewards" in turn to update the selector. We build the reinforced data selector based on the actor-critic framework and integrate it to a DNN based transfer learning model, resulting in a Reinforced Transfer Learning (RTL) method. We perform a thorough experimental evaluation on two major tasks for text matching, namely, paraphrase identification and natural language inference. Experimental results show the proposed RTL can significantly improve the performance of the TL model. We further investigate different settings of states, rewards, and policy optimization methods to examine the robustness of our method. Last, we conduct a case study on the selected data and find our method is able to select source domain data whose Wasserstein distance is close to the target domain data. This is reasonable and intuitive as such source domain data can provide more transferability power to the model.


page 1

page 2

page 3

page 4


Cross-position Activity Recognition with Stratified Transfer Learning

Human activity recognition aims to recognize the activities of daily liv...

A Transfer Learning Pipeline for Educational Resource Discovery with Application in Leading Paragraph Generation

Effective human learning depends on a wide selection of educational mate...

Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce

In this paper, we study transfer learning for the PI and NLI problems, a...

Comparison of Transfer Learning based Additive Manufacturing Models via A Case Study

Transfer learning (TL) based additive manufacturing (AM) modeling is an ...

Detecting Bias in Transfer Learning Approaches for Text Classification

Classification is an essential and fundamental task in machine learning,...

The Multiverse Loss for Robust Transfer Learning

Deep learning techniques are renowned for supporting effective transfer ...

Privacy-preserving Transfer Learning via Secure Maximum Mean Discrepancy

The success of machine learning algorithms often relies on a large amoun...