DeepAI
Log In Sign Up

From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding

05/15/2021
by   Rob van der Goot, et al.
0

The lack of publicly available evaluation data for low-resource languages limits progress in Spoken Language Understanding (SLU). As key tasks like intent classification and slot filling require abundant training data, it is desirable to reuse existing data in high-resource languages to develop models for low-resource scenarios. We introduce xSID, a new benchmark for cross-lingual Slot and Intent Detection in 13 languages from 6 language families, including a very low-resource dialect. To tackle the challenge, we propose a joint learning approach, with English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer. We study two setups which differ by type and language coverage of the pre-trained embeddings. Our results show that jointly learning the main tasks with masked language modeling is effective for slots, while machine translation transfer works best for intent classification.

READ FULL TEXT

page 3

page 8

10/18/2021

Intent Classification Using Pre-Trained Embeddings For Low Resource Languages

Building Spoken Language Understanding (SLU) systems that do not rely on...
04/17/2021

Multilingual and Cross-Lingual Intent Detection from Spoken Data

We present a systematic study on multilingual and cross-lingual intent d...
09/03/2021

Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding

Lack of training data presents a grand challenge to scaling out spoken l...
11/15/2019

Bootstrapping NLU Models with Multi-task Learning

Bootstrapping natural language understanding (NLU) systems with minimal ...
07/12/2021

End-to-End Natural Language Understanding Pipeline for Bangla Conversational Agents

Chatbots are intelligent software built to be used as a replacement for ...
08/06/2020

Data balancing for boosting performance of low-frequency classes in Spoken Language Understanding

Despite the fact that data imbalance is becoming more and more common in...