Uralic Language Identification (ULI) 2020 shared task dataset and the Wanca 2017 corpus

08/27/2020
by   Tommi Jauhiainen, et al.
0

This article introduces the Wanca 2017 corpus of texts crawled from the internet from which the sentences in rare Uralic languages for the use of the Uralic Language Identification (ULI) 2020 shared task were collected. We describe the ULI dataset and how it was constructed using the Wanca 2017 corpus and texts in different languages from the Leipzig corpora collection. We also provide baseline language identification experiments conducted using the ULI 2020 dataset.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

03/05/2019

Language and Dialect Identification of Cuneiform Texts

This article introduces a corpus of cuneiform texts from which the datas...
06/14/2018

Automatic Language Identification for Romance Languages using Stop Words and Diacritics

Automatic language identification is a natural language processing probl...
06/10/2019

AGRR-2019: A Corpus for Gapping Resolution in Russian

This paper provides a comprehensive overview of the gapping dataset for ...
10/28/2020

Character Entropy in Modern and Historical Texts: Comparison Metrics for an Undeciphered Manuscript

This paper outlines the creation of three corpora for multilingual compa...
01/17/2022

PerPaDa: A Persian Paraphrase Dataset based on Implicit Crowdsourcing Data Collection

In this paper we introduce PerPaDa, a Persian paraphrase dataset that is...
07/16/2017

Open-Set Language Identification

We present the first open-set language identification experiments using ...
05/30/2017

A Low Dimensionality Representation for Language Variety Identification

Language variety identification aims at labelling texts in a native lang...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.