On Language Models for Creoles

09/13/2021
by   Heather Lent, et al.
23

Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature. Creoles typically result from the fusion of a foreign language with multiple local languages, and what grammatical and lexical features are transferred to the creole is a complex process. While creoles are generally stable, the prominence of some features may be much stronger with certain demographics or in some linguistic situations. This paper makes several contributions: We collect existing corpora and release models for Haitian Creole, Nigerian Pidgin English, and Singaporean Colloquial English. We evaluate these models on intrinsic and extrinsic tasks. Motivated by the above literature, we compare standard language models with distributionally robust ones and find that, somewhat surprisingly, the standard language models are superior to the distributionally robust ones. We investigate whether this is an effect of over-parameterization or relative distributional stability, and find that the difference persists in the absence of over-parameterization, and that drift is limited, confirming the relative stability of creole languages.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2020

Word Frequency Does Not Predict Grammatical Knowledge in Language Models

Neural language models learn, to varying degrees of accuracy, the gramma...
research
11/23/2021

Using Distributional Principles for the Semantic Study of Contextual Language Models

Many studies were recently done for investigating the properties of cont...
research
02/28/2021

RuSentEval: Linguistic Source, Encoder Force!

The success of pre-trained transformer language models has brought a gre...
research
12/30/2020

Predicting cross-linguistic adjective order with information gain

Languages vary in their placement of multiple adjectives before, after, ...
research
12/10/2020

As good as new. How to successfully recycle English GPT-2 to make models for other languages

Large generative language models have been very successful for English, ...
research
12/11/2022

IndicXTREME: A Multi-Task Benchmark For Evaluating Indic Languages

In this work, we introduce IndicXTREME, a benchmark consisting of nine d...
research
09/20/2022

Register Variation Remains Stable Across 60 Languages

This paper measures the stability of cross-linguistic register variation...

Please sign up or login with your details

Forgot password? Click here to reset