DeepAI AI Chat
Log In Sign Up

Borrowing or Codeswitching? Annotating for Finer-Grained Distinctions in Language Mixing

by   Elena Alvarez-Mellado, et al.
Brandeis University

We present a new corpus of Twitter data annotated for codeswitching and borrowing between Spanish and English. The corpus contains 9,500 tweets annotated at the token level with codeswitches, borrowings, and named entities. This corpus differs from prior corpora of codeswitching in that we attempt to clearly define and annotate the boundary between codeswitching and borrowing and do not treat common "internet-speak" ('lol', etc.) as codeswitching when used in an otherwise monolingual context. The result is a corpus that enables the study and modeling of Spanish-English borrowing and codeswitching on Twitter in one dataset. We present baseline scores for modeling the labels of this corpus using Transformer-based language models. The annotation itself is released with a CC BY 4.0 license, while the text it applies to is distributed in compliance with the Twitter terms of service.


page 1

page 2

page 3

page 4


Gibberish Semantics: How Good is Russian Twitter in Word Semantic Similarity Task?

The most studied and most successful language models were developed and ...

A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers

We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in te...

Aggression-annotated Corpus of Hindi-English Code-mixed Data

As the interaction over the web has increased, incidents of aggression a...

Extracting COVID-19 Events from Twitter

We present a corpus of 7,500 tweets annotated with COVID-19 events, incl...

Twitter Job/Employment Corpus: A Dataset of Job-Related Discourse Built with Humans in the Loop

We present the Twitter Job/Employment Corpus, a collection of tweets ann...

A corpus of preposition supersenses in English web reviews

We present the first corpus annotated with preposition supersenses, unle...

TimeLMs: Diachronic Language Models from Twitter

Despite its importance, the time variable has been largely neglected in ...