mcBERT: Momentum Contrastive Learning with BERT for Zero-Shot Slot Filling

03/24/2022
by   Seong-Hwan Heo, et al.
0

Zero-shot slot filling has received considerable attention to cope with the problem of limited available data for the target domain. One of the important factors in zero-shot learning is to make the model learn generalized and reliable representations. For this purpose, we present mcBERT, which stands for momentum contrastive learning with BERT, to develop a robust zero-shot slot filling model. mcBERT uses BERT to initialize the two encoders, the query encoder and key encoder, and is trained by applying momentum contrastive learning. Our experimental results on the SNIPS benchmark show that mcBERT substantially outperforms the previous models, recording a new state-of-the-art. Besides, we also show that each component composing mcBERT contributes to the performance improvement.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/07/2021

Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling

Zero-shot cross-domain slot filling alleviates the data dependence in th...
research
07/06/2023

Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting

Zero-shot cross-domain slot filling aims to transfer knowledge from the ...
research
11/24/2020

Zero-Shot Visual Slot Filling as Question Answering

This paper presents a new approach to visual zero-shot slot filling. The...
research
06/07/2023

Prompter: Zero-shot Adaptive Prefixes for Dialogue State Tracking Domain Adaptation

A challenge in the Dialogue State Tracking (DST) field is adapting model...
research
07/04/2023

Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Data

Manually annotating fine-grained slot-value labels for task-oriented dia...
research
03/24/2023

Toward Open-domain Slot Filling via Self-supervised Co-training

Slot filling is one of the critical tasks in modern conversational syste...
research
02/27/2023

Revisit Out-Of-Vocabulary Problem for Slot Filling: A Unified Contrastive Frameword with Multi-level Data Augmentations

In real dialogue scenarios, the existing slot filling model, which tends...

Please sign up or login with your details

Forgot password? Click here to reset