Sebastian Ruder

research

∙ 07/11/2023

Empowering Cross-lingual Behavioral Testing of NLP Models with Typological Features

A challenge towards developing NLP systems for the world's languages is ...

0 Ester Hlavnova, et al. ∙

research

∙ 05/24/2023

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer

Despite remarkable advancements in few-shot generalization in natural la...

0 Akari Asai, et al. ∙

research

∙ 05/23/2023

Evaluating and Modeling Attribution for Cross-Lingual Question Answering

Trustworthy answer content is abundant in many high-resource languages a...

0 Benjamin Müller, et al. ∙

research

∙ 05/23/2023

mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations

Multilingual sequence-to-sequence models perform poorly with increased l...

0 Jonas Pfeiffer, et al. ∙

research

∙ 05/19/2023

XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages

Data scarcity is a crucial issue for the development of highly multiling...

0 Sebastian Ruder, et al. ∙

research

∙ 04/18/2023

Romanization-based Large-scale Adaptation of Multilingual Language Models

Large multilingual pretrained language models (mPLMs) have become the de...

0 Sukannya Purkayastha, et al. ∙

research

∙ 02/22/2023

Modular Deep Learning

Transfer learning has recently become the dominant paradigm of machine l...

0 Jonas Pfeiffer, et al. ∙

research

∙ 02/17/2023

AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages

Africa is home to over 2000 languages from over six language families an...

11 Shamsuddeen Hassan Muhammad, et al. ∙

research

∙ 12/19/2022

NusaCrowd: Open Source Initiative for Indonesian NLP Resources

We present NusaCrowd, a collaborative initiative to collect and unite ex...

0 Samuel Cahyawijaya, et al. ∙

research

∙ 11/15/2022

QAmeleon: Multilingual QA with Only 5 Examples

The availability of large, high-quality datasets has been one of the mai...

0 Priyanka Agrawal, et al. ∙

research

∙ 10/31/2022

TaTa: A Multilingual Table-to-Text Dataset for African Languages

Existing data-to-text generation datasets are mostly limited to English....

0 Sebastian Gehrmann, et al. ∙

research

∙ 10/06/2022

Language Models are Multilingual Chain-of-Thought Reasoners

We evaluate the reasoning abilities of large language models in multilin...

0 Freda Shi, et al. ∙

research

∙ 06/20/2022

Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifold

The prototypical NLP experiment trains a standard architecture on labele...

0 Sebastian Ruder, et al. ∙

research

∙ 05/31/2022

NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages

Natural language processing (NLP) has a significant impact on society vi...

5 Genta Indra Winata, et al. ∙

research

∙ 05/25/2022

Evaluating Inclusivity, Equity, and Accessibility of NLP Technology: A Case Study for Indian Languages

In order for NLP technology to be widely applicable and useful, it needs...

0 Simran Khanuja, et al. ∙

research

∙ 05/24/2022

Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer

Massively multilingual models are promising for transfer learning across...

0 Ahmet Üstün, et al. ∙

research

∙ 03/24/2022

One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia

NLP research is impeded by a lack of resources and awareness of the chal...

4 Alham Fikri Aji, et al. ∙

research

∙ 03/21/2022

XTREME-S: Evaluating Cross-lingual Speech Representations

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingu...

0 Alexis Conneau, et al. ∙

research

∙ 03/17/2022

Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation

The performance of multilingual pretrained models is highly dependent on...

0 Xinyi Wang, et al. ∙

research

∙ 01/20/2022

NaijaSenti: A Nigerian Twitter Sentiment Corpus for Multilingual Sentiment Analysis

Sentiment analysis is one of the most widely studied applications in NLP...

11 Shamsuddeen Hassan Muhammad, et al. ∙

research

∙ 10/12/2021

Balancing Average and Worst-case Accuracy in Multitask Learning

When training and evaluating machine learning models on a large number o...

0 Paul Michel, et al. ∙

research

∙ 09/27/2021

FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding

The few-shot natural language understanding (NLU) task has attracted muc...

8 Yanan Zheng, et al. ∙

research

∙ 09/10/2021

Efficient Test Time Adapter Ensembling for Low-resource Language Varieties

Adapters are light-weight modules that allow parameter-efficient fine-tu...

0 Xinyi Wang, et al. ∙

research

∙ 06/23/2021

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization

State-of-the-art models in natural language processing rely on separate ...

4 Yi Tay, et al. ∙

research

∙ 06/08/2021

Compacter: Efficient Low-Rank Hypercomplex Adapter Layers

Adapting large-scale pretrained language models to downstream tasks via ...

0 Rabeeh Karimi Mahabadi, et al. ∙

research

∙ 06/08/2021

Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks

State-of-the-art parameter-efficient fine-tuning methods rely on introdu...

0 Rabeeh Karimi Mahabadi, et al. ∙

research

∙ 04/16/2021

IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation

A benchmark provides an ecosystem to measure the advancement of models w...

6 Samuel Cahyawijaya, et al. ∙

research

∙ 04/15/2021

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation

Machine learning has brought striking advances in multilingual natural l...

0 Sebastian Ruder, et al. ∙

research

∙ 03/22/2021

MasakhaNER: Named Entity Recognition for African Languages

We take a step towards addressing the under-representation of the Africa...

5 David Ifeoluwa Adelani, et al. ∙

research

∙ 03/15/2021

Multi-view Subword Regularization

Multilingual pretrained representations generally rely on subword segmen...

0 Xinyi Wang, et al. ∙

research

∙ 02/03/2021

Pitfalls of Static Language Modelling

Our world is open-ended, non-stationary and constantly evolving; thus wh...

0 Angeliki Lazaridou, et al. ∙

research

∙ 12/31/2020

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

In this work we provide a systematic empirical comparison of pretrained ...

12 Phillip Rust, et al. ∙

research

∙ 12/31/2020

UNKs Everywhere: Adapting Multilingual Language Models to New Scripts

Massively multilingual language models such as multilingual BERT (mBERT)...

4 Jonas Pfeiffer, et al. ∙

research

∙ 11/15/2020

Morphologically Aware Word-Level Translation

We propose a novel morphologically aware probability model for bilingual...

0 Paula Czarnowska, et al. ∙

research

∙ 11/08/2020

Long Range Arena: A Benchmark for Efficient Transformers

Transformers do not scale very well to long sequence lengths largely bec...

5 Yi Tay, et al. ∙

research

∙ 10/24/2020

Rethinking embedding coupling in pre-trained language models

We re-evaluate the standard practice of sharing weights between input an...

0 Hyung Won Chung, et al. ∙

research

∙ 07/15/2020

AdapterHub: A Framework for Adapting Transformers

The current modus operandi in NLP involves downloading and fine-tuning p...

0 Jonas Pfeiffer, et al. ∙

research

∙ 04/30/2020

MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer

The main goal behind state-of-the-art pretrained multilingual models suc...

0 Jonas Pfeiffer, et al. ∙

research

∙ 04/30/2020

A Call for More Rigor in Unsupervised Cross-lingual Learning

We review motivations, definition, approaches, and methodology for unsup...

6 Mikel Artetxe, et al. ∙

research

∙ 04/29/2020

AxCell: Automatic Extraction of Results from Machine Learning Papers

Tracking progress in machine learning has become increasingly difficult ...

2 Marcin Kardas, et al. ∙

research

∙ 04/08/2020

Are All Good Word Vector Spaces Isomorphic?

Existing algorithms for aligning cross-lingual word vector spaces assume...

0 Ivan Vulić, et al. ∙

research

∙ 03/24/2020

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

Much recent progress in applications of machine learning models to NLP h...

0 Junjie Hu, et al. ∙

research

∙ 10/25/2019

On the Cross-lingual Transferability of Monolingual Representations

State-of-the-art unsupervised multilingual models (e.g., multilingual BE...

14 Mikel Artetxe, et al. ∙

research

∙ 09/10/2019

MultiFiT: Efficient Multi-lingual Language Model Fine-tuning

Pretrained language models are promising particularly for low-resource l...

0 Julian Eisenschlos, et al. ∙

research

∙ 09/10/2019

What do Deep Networks Like to Read?

Recent research towards understanding neural networks probes models in a...

0 Jonas Pfeiffer, et al. ∙

research

∙ 09/06/2019

Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Human translators routinely have to translate rare inflections of words ...

0 Paula Czarnowska, et al. ∙

research

∙ 06/03/2019

Episodic Memory in Lifelong Language Learning

We introduce a lifelong language learning setup where a model needs to l...

8 Cyprien de Masson d'Autume, et al. ∙

research

∙ 03/14/2019

To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks

While most previous work has focused on different pretraining objectives...

0 Matthew Peters, et al. ∙

research

∙ 02/01/2019

How to (Properly) Evaluate Cross-Lingual Word Embeddings: On Strong Baselines, Comparative Analyses, and Some Misconceptions

Cross-lingual word embeddings (CLEs) enable multilingual modeling of mea...

0 Goran Glavaš, et al. ∙

research

∙ 11/14/2018

A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks

Much efforts has been devoted to evaluate whether multi-task learning ca...

0 Victor Sanh, et al. ∙

Sebastian Ruder

Featured Co-authors