Language Models are Few-shot Multilingual Learners

09/16/2021
by   Genta Indra Winata, et al.
6

General-purpose language models have demonstrated impressive capabilities, performing on par with state-of-the-art approaches on a range of downstream natural language processing (NLP) tasks and benchmarks when inferring instructions from very few examples. Here, we evaluate the multilingual skills of the GPT and T5 models in conducting multi-class classification on non-English languages without any parameter updates. We show that, given a few English examples as context, pre-trained language models can predict not only English test samples but also non-English ones. Finally, we find the in-context few-shot cross-lingual prediction results of language models are significantly better than random prediction, and they are competitive compared to the existing state-of-the-art cross-lingual models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2023

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer

Despite remarkable advancements in few-shot generalization in natural la...
research
05/03/2022

XLTime: A Cross-Lingual Knowledge Transfer Framework for Temporal Expression Extraction

Temporal Expression Extraction (TEE) is essential for understanding time...
research
12/20/2021

Few-shot Learning with Multilingual Language Models

Large-scale autoregressive language models such as GPT-3 are few-shot le...
research
06/19/2023

Multilingual Few-Shot Learning via Language Model Retrieval

Transformer-based language models have achieved remarkable success in fe...
research
04/22/2022

Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks

Logical approaches to representing language have developed and evaluated...
research
09/28/2021

Multilingual Counter Narrative Type Classification

The growing interest in employing counter narratives for hatred interven...
research
09/15/2021

On the Universality of Deep COntextual Language Models

Deep Contextual Language Models (LMs) like ELMO, BERT, and their success...

Please sign up or login with your details

Forgot password? Click here to reset