Zero-Shot Visual Slot Filling as Question Answering

11/24/2020
by   Larry Heck, et al.
0

This paper presents a new approach to visual zero-shot slot filling. The approach extends previous approaches by reformulating the slot filling task as Question Answering. Slot tags are converted to rich natural language questions that capture the semantics of visual information and lexical text on the GUI screen. These questions are paired with the user's utterance and slots are extracted from the utterance using a state-of-the-art ALBERT-based Question Answering system trained on the Stanford Question Answering dataset (SQuaD2). An approach to further refine the model with multi-task training is presented. The multi-task approach facilitates the incorporation of a large number of successive refinements and transfer learning across similar tasks. A new Visual Slot dataset and a visual extension of the popular ATIS dataset is introduced to support research and experimentation on visual slot filling. Results show F1 scores between 0.52 and 0.60 on the Visual Slot and ATIS datasets with no training data (zero-shot).

READ FULL TEXT

page 2

page 3

research
04/22/2018

Question Answering Resources Applied to Slot-Filling

We investigate the utility of pre-existing question answering models and...
research
09/07/2023

Introducing "Forecast Utterance" for Conversational Data Science

Envision an intelligent agent capable of assisting users in conducting f...
research
01/16/2021

Linguistically-Enriched and Context-Aware Zero-shot Slot Filling

Slot filling is identifying contiguous spans of words in an utterance th...
research
01/20/2021

Zero-shot Generalization in Dialog State Tracking through Generative Question Answering

Dialog State Tracking (DST), an integral part of modern dialog systems, ...
research
03/24/2022

mcBERT: Momentum Contrastive Learning with BERT for Zero-Shot Slot Filling

Zero-shot slot filling has received considerable attention to cope with ...
research
07/04/2023

Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Data

Manually annotating fine-grained slot-value labels for task-oriented dia...
research
06/13/2021

GenSF: Simultaneous Adaptation of Generative Pre-trained Models and Slot Filling

In transfer learning, it is imperative to achieve strong alignment betwe...

Please sign up or login with your details

Forgot password? Click here to reset