Transfer Learning for Sequence Labeling Using Source Model and Target Data

02/14/2019
by   Lingzhen Chen, et al.
0

In this paper, we propose an approach for transferring the knowledge of a neural model for sequence labeling, learned from the source domain, to a new model trained on a target domain, where new label categories appear. Our transfer learning (TL) techniques enable to adapt the source model using the target data and new categories, without accessing to the source data. Our solution consists in adding new neurons in the output layer of the target model and transferring parameters from the source model, which are then fine-tuned with the target data. Additionally, we propose a neural adapter to learn the difference between the source and the target label distribution, which provides additional important information to the target model. Our experiments on Named Entity Recognition show that (i) the learned knowledge in the source model can be effectively transferred when the target data contains new categories and (ii) our neural adapter further improves such transfer.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/18/2021

A new semi-supervised inductive transfer learning framework: Co-Transfer

In many practical data mining scenarios, such as network intrusion detec...
research
08/19/2019

Transfer Learning-Based Label Proportions Method with Data of Uncertainty

Learning with label proportions (LLP), which is a learning task that onl...
research
11/29/2022

Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer

When presented with a data stream of two statistically dependent variabl...
research
10/13/2021

An Efficient Source Model Selection Framework in Model Databases

With the explosive increase of big data, training a Machine Learning (ML...
research
02/25/2019

Transfer Learning for Sequences via Learning to Collocate

Transfer learning aims to solve the data sparsity for a target domain by...
research
01/18/2021

Transferring model structure in Bayesian transfer learning for Gaussian process regression

Bayesian transfer learning (BTL) is defined in this paper as the task of...
research
05/12/2022

Target Aware Network Architecture Search and Compression for Efficient Knowledge Transfer

Transfer Learning enables Convolutional Neural Networks (CNN) to acquire...

Please sign up or login with your details

Forgot password? Click here to reset