GIRNet: Interleaved Multi-Task Recurrent State Sequence Models

11/28/2018
by   Divam Gupta, et al.
0

In several natural language tasks, labeled sequences are available in separate domains (say, languages), but the goal is to label sequences with mixed domain (such as code-switched text). Or, we may have available models for labeling whole passages (say, with sentiments), which we would like to exploit toward better position-specific label inference (say, target-dependent sentiment annotation). A key characteristic shared across such tasks is that different positions in a primary instance can benefit from different `experts' trained from auxiliary data, but labeled primary instances are scarce, and labeling the best expert for each position entails unacceptable cognitive burden. We propose GITNet, a unified position-sensitive multi-task recurrent neural network (RNN) architecture for such applications. Auxiliary and primary tasks need not share training instances. Auxiliary RNNs are trained over auxiliary instances. A primary instance is also submitted to each auxiliary RNN, but their state sequences are gated and merged into a novel composite state sequence tailored to the primary inference task. Our approach is in sharp contrast to recent multi-task networks like the cross-stitch and sluice network, which do not control state transfer at such fine granularity. We demonstrate the superiority of GIRNet using three applications: sentiment classification of code-switched passages, part-of-speech tagging of code-switched text, and target position-sensitive annotation of sentiment in monolingual passages. In all cases, we establish new state-of-the-art performance beyond recent competitive baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2019

Multi-task Learning for Target-dependent Sentiment Classification

Detecting and aggregating sentiments toward people, organizations, and e...
research
02/27/2018

Multi-task Learning of Pairwise Sequence Classification Tasks Over Disparate Label Spaces

We combine multi-task learning and semi-supervised learning by inducing ...
research
10/13/2022

Composite Learning for Robust and Effective Dense Predictions

Multi-task learning promises better model generalization on a target tas...
research
05/19/2020

On the Choice of Auxiliary Languages for Improved Sequence Tagging

Recent work showed that embeddings from related languages can improve th...
research
06/14/2017

Transfer Learning for Neural Semantic Parsing

The goal of semantic parsing is to map natural language to a machine int...
research
06/13/2019

Improved Sentiment Detection via Label Transfer from Monolingual to Synthetic Code-Switched Text

Multilingual writers and speakers often alternate between two languages ...
research
11/10/2020

Multi-Task Sequence Prediction For Tunisian Arabizi Multi-Level Annotation

In this paper we propose a multi-task sequence prediction system, based ...

Please sign up or login with your details

Forgot password? Click here to reset