FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding

by   Yutai Hou, et al.

Few-learn learning (FSL) is one of the key future steps in machine learning and has raised a lot of attention. However, in contrast to the rapid development in other domains, such as Computer Vision, the progress of FSL in Nature Language Processing (NLP) is much slower. One of the key reasons for this is the lacking of public benchmarks. NLP FSL researches always report new results on their own constructed few-shot datasets, which is pretty inefficient in results comparison and thus impedes cumulative progress. In this paper, we present FewJoint, a novel Few-Shot Learning benchmark for NLP. Different from most NLP FSL research that only focus on simple N-classification problems, our benchmark introduces few-shot joint dialogue language understanding, which additionally covers the structure prediction and multi-task reliance problems. This allows our benchmark to reflect the real-word NLP complexity beyond simple N-classification. Our benchmark is used in the few-shot learning contest of SMP2020-ECDT task-1. We also provide a compatible FSL platform to ease experiment set-up.



There are no comments yet.


page 1

page 2

page 3

page 4


One-Shot Learning for Language Modelling

Humans can infer a great deal about the meaning of a word, using the syn...

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Most recent progress in natural language understanding (NLU) has been dr...

FLEX: Unifying Evaluation for Few-Shot NLP

Few-shot NLP research is highly active, yet conducted in disjoint resear...

Evaluating NLP Systems On a Novel Cloze Task: Judging the Plausibility of Possible Fillers in Instructional Texts

Cloze task is a widely used task to evaluate an NLP system's language un...

Learning to Control Latent Representations for Few-Shot Learning of Named Entities

Humans excel in continuously learning with small data without forgetting...

Few-shot learning for medical text: A systematic review

Objective: Few-shot learning (FSL) methods require small numbers of labe...

The Omniglot Challenge: A 3-Year Progress Report

Three years ago, we released the Omniglot dataset for developing more hu...

Code Repositories



view repo


Platform for few-shot natural language processing: Text Classification, Sequene Labeling.

view repo


datasets of natural language understanding and dialogue state tracking

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.