Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks

03/18/2017
by   Zhilin Yang, et al.
0

Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks. One appealing property of such systems is their generality, as excellent performance can be achieved with a unified architecture and without task-specific feature engineering. However, it is unclear if such systems can be used for tasks without large amounts of training data. In this paper we explore the problem of transfer learning for neural sequence taggers, where a source task with plentiful annotations (e.g., POS tagging on Penn Treebank) is used to improve performance on a target task with fewer available annotations (e.g., POS tagging for microblogs). We examine the effects of transfer learning for deep hierarchical recurrent networks across domains, applications, and languages, and show that significant improvement can often be obtained. These improvements lead to improvements over the current state-of-the-art on several well-studied tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/01/2015

A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding

Bidirectional Long Short-Term Memory Recurrent Neural Network (BLSTM-RNN...
research
12/26/2018

A New Concept of Deep Reinforcement Learning based Augmented General Sequence Tagging System

In this paper, a new deep reinforcement learning based augmented general...
research
03/20/2016

Multi-Task Cross-Lingual Sequence Tagging from Scratch

We present a deep hierarchical recurrent neural network for sequence tag...
research
05/20/2022

Deep transfer learning for image classification: a survey

Deep neural networks such as convolutional neural networks (CNNs) and tr...
research
07/13/2019

Cross-Lingual Transfer Learning for Question Answering

Deep learning based question answering (QA) on English documents has ach...
research
03/04/2016

Integrated Sequence Tagging for Medieval Latin Using Deep Representation Learning

In this paper we consider two sequence tagging tasks for medieval Latin:...
research
03/11/2022

Leveraging universality of jet taggers through transfer learning

A significant challenge in the tagging of boosted objects via machine-le...

Please sign up or login with your details

Forgot password? Click here to reset