Augmented Natural Language for Generative Sequence Labeling

by   Ben Athiwaratkun, et al.

We propose a generative framework for joint sequence labeling and sentence-level classification. Our model performs multiple sequence labeling tasks at once using a single, shared natural language output space. Unlike prior discriminative methods, our model naturally incorporates label semantics and shares knowledge across tasks. Our framework is general purpose, performing well on few-shot, low-resource, and high-resource tasks. We demonstrate these advantages on popular named entity recognition, slot labeling, and intent classification benchmarks. We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot (75.0%→ 90.9%) and 1-shot (70.4%→ 81.0%) state-of-the-art results. Furthermore, our model generates large improvements (46.27%→ 63.83%) in low-resource slot labeling over a BERT baseline by incorporating label semantics. We also maintain competitive results on high-resource tasks, performing within two points of the state-of-the-art on all tasks and setting a new state-of-the-art on the SNIPS dataset.


Label Semantics for Few Shot Named Entity Recognition

We study the problem of few shot learning for named entity recognition. ...

Structured Prediction as Translation between Augmented Natural Languages

We propose a new framework, Translation between Augmented Natural Langua...

Jointly Trained Sequential Labeling and Classification by Sparse Attention Neural Networks

Sentence-level classification and sequential labeling are two fundamenta...

InstructionNER: A Multi-Task Instruction-Based Generative Framework for Few-shot NER

Recently, prompt-based methods have achieved significant performance in ...

Few-Shot Sequence Labeling with Label Dependency Transfer

Few-shot sequence labeling faces a unique challenge compared with the ot...

SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup

Active learning is an important technique for low-resource sequence labe...

How Does Beam Search improve Span-Level Confidence Estimation in Generative Sequence Labeling?

Text-to-text generation models have increasingly become the go-to soluti...

Please sign up or login with your details

Forgot password? Click here to reset