FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding

09/17/2020
by   Yutai Hou, et al.
10

Few-learn learning (FSL) is one of the key future steps in machine learning and has raised a lot of attention. However, in contrast to the rapid development in other domains, such as Computer Vision, the progress of FSL in Nature Language Processing (NLP) is much slower. One of the key reasons for this is the lacking of public benchmarks. NLP FSL researches always report new results on their own constructed few-shot datasets, which is pretty inefficient in results comparison and thus impedes cumulative progress. In this paper, we present FewJoint, a novel Few-Shot Learning benchmark for NLP. Different from most NLP FSL research that only focus on simple N-classification problems, our benchmark introduces few-shot joint dialogue language understanding, which additionally covers the structure prediction and multi-task reliance problems. This allows our benchmark to reflect the real-word NLP complexity beyond simple N-classification. Our benchmark is used in the few-shot learning contest of SMP2020-ECDT task-1. We also provide a compatible FSL platform to ease experiment set-up.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/19/2020

One-Shot Learning for Language Modelling

Humans can infer a great deal about the meaning of a word, using the syn...
11/04/2021

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Most recent progress in natural language understanding (NLU) has been dr...
07/15/2021

FLEX: Unifying Evaluation for Few-Shot NLP

Few-shot NLP research is highly active, yet conducted in disjoint resear...
12/03/2021

Evaluating NLP Systems On a Novel Cloze Task: Judging the Plausibility of Possible Fillers in Instructional Texts

Cloze task is a widely used task to evaluate an NLP system's language un...
11/19/2019

Learning to Control Latent Representations for Few-Shot Learning of Named Entities

Humans excel in continuously learning with small data without forgetting...
04/21/2022

Few-shot learning for medical text: A systematic review

Objective: Few-shot learning (FSL) methods require small numbers of labe...
02/09/2019

The Omniglot Challenge: A 3-Year Progress Report

Three years ago, we released the Omniglot dataset for developing more hu...

Code Repositories

ChineseNLPCorpus

中文自然语言处理数据集,平时做做实验的材料。欢迎补充提交合并。


view repo

MetaDialog

Platform for few-shot natural language processing: Text Classification, Sequene Labeling.


view repo

NLU_datasets_with_task_oriented_dialogue

datasets of natural language understanding and dialogue state tracking


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.