Is Machine Learning Speaking my Language? A Critical Look at the NLP-Pipeline Across 8 Human Languages

by   Esma Wali, et al.

Natural Language Processing (NLP) is increasingly used as a key ingredient in critical decision-making systems such as resume parsers used in sorting a list of job candidates. NLP systems often ingest large corpora of human text, attempting to learn from past human behavior and decisions in order to produce systems that will make recommendations about our future world. Over 7000 human languages are being spoken today and the typical NLP pipeline underrepresents speakers of most of them while amplifying the voices of speakers of other languages. In this paper, a team including speakers of 8 languages - English, Chinese, Urdu, Farsi, Arabic, French, Spanish, and Wolof - takes a critical look at the typical NLP pipeline and how even when a language is technically supported, substantial caveats remain to prevent full participation. Despite huge and admirable investments in multilingual support in many tools and resources, we are still making NLP-guided decisions that systematically and dramatically underrepresent the voices of much of the world.



There are no comments yet.


page 1

page 2

page 3

page 4


A Panoramic Survey of Natural Language Processing in the Arab World

The term natural language refers to any system of symbolic communication...

One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia

NLP research is impeded by a lack of resources and awareness of the chal...

An assessment of orthographic similarity measures for several African languages

Natural Language Interfaces and tools such as spellcheckers and Web sear...

Challenges and Strategies in Cross-Cultural NLP

Various efforts in the Natural Language Processing (NLP) community have ...

Masader: Metadata Sourcing for Arabic Text and Speech Data Resources

The NLP pipeline has evolved dramatically in the last few years. The fir...

Writing Across the World's Languages: Deep Internationalization for Gboard, the Google Keyboard

This technical report describes our deep internationalization program fo...

Investigating Effect of Dialogue History in Multilingual Task Oriented Dialogue Systems

While the English virtual assistants have achieved exciting performance ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.