DeepAI AI Chat
Log In Sign Up

MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging

by   Hongru Wang, et al.

Meta-learning is widely used for few-shot slot tagging in the task of few-shot learning. The performance of existing methods is, however, seriously affected by catastrophic forgetting. This phenomenon is common in deep learning as the training and testing modules fail to take into account historical information, i.e. previously trained episodes in the metric-based meta-learning. To overcome this predicament, we propose the Memory-based Contrastive Meta-learning (MCML) method. Specifically, we propose a learn-from-memory mechanism that use explicit memory to keep track of the label representations of previously trained episodes and propose a contrastive learning method to compare the current label embedded in the few shot episode with the historic ones stored in the memory, and an adaption-from memory mechanism to determine the output label based on the contrast between the input labels embedded in the test episode and the label clusters in the memory. Experimental results show that MCML is scalable and outperforms metric-based meta-learning and optimization-based meta-learning on all 1shot, 5-shot, 10-shot, and 20-shot scenarios of the SNIPS dataset.


page 1

page 2

page 3

page 4


Graph Contrastive Learning Meets Graph Meta Learning: A Unified Method for Few-shot Node Tasks

Graph Neural Networks (GNNs) have become popular in Graph Representation...

Meta-Learning via Feature-Label Memory Network

Deep learning typically requires training a very capable architecture us...

A contrastive learning approach for individual re-identification in a wild fish population

In both terrestrial and marine ecology, physical tagging is a frequently...

Learning to Learn with Indispensable Connections

Meta-learning aims to solve unseen tasks with few labelled instances. Ne...

Contrastive Meta-Learning for Partially Observable Few-Shot Learning

Many contrastive and meta-learning approaches learn representations by i...

Meta Learning with Relational Information for Short Sequences

This paper proposes a new meta-learning method -- named HARMLESS (HAwkes...

BatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noise

The negative impact of label noise is well studied in classical supervis...