Borrowing or Codeswitching? Annotating for Finer-Grained Distinctions in Language Mixing

06/10/2022
by   Elena Alvarez-Mellado, et al.
0

We present a new corpus of Twitter data annotated for codeswitching and borrowing between Spanish and English. The corpus contains 9,500 tweets annotated at the token level with codeswitches, borrowings, and named entities. This corpus differs from prior corpora of codeswitching in that we attempt to clearly define and annotate the boundary between codeswitching and borrowing and do not treat common "internet-speak" ('lol', etc.) as codeswitching when used in an otherwise monolingual context. The result is a corpus that enables the study and modeling of Spanish-English borrowing and codeswitching on Twitter in one dataset. We present baseline scores for modeling the labels of this corpus using Transformer-based language models. The annotation itself is released with a CC BY 4.0 license, while the text it applies to is distributed in compliance with the Twitter terms of service.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/28/2016

Gibberish Semantics: How Good is Russian Twitter in Word Semantic Similarity Task?

The most studied and most successful language models were developed and ...
research
06/30/2021

A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers

We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in te...
research
06/24/2023

My Boli: Code-mixed Marathi-English Corpora, Pretrained Language Models and Evaluation Benchmarks

The research on code-mixed data is limited due to the unavailability of ...
research
03/26/2018

Aggression-annotated Corpus of Hindi-English Code-mixed Data

As the interaction over the web has increased, incidents of aggression a...
research
01/30/2019

Twitter Job/Employment Corpus: A Dataset of Job-Related Discourse Built with Humans in the Loop

We present the Twitter Job/Employment Corpus, a collection of tweets ann...
research
05/08/2016

A corpus of preposition supersenses in English web reviews

We present the first corpus annotated with preposition supersenses, unle...
research
02/08/2022

TimeLMs: Diachronic Language Models from Twitter

Despite its importance, the time variable has been largely neglected in ...

Please sign up or login with your details

Forgot password? Click here to reset