DeepAI AI Chat
Log In Sign Up

Leveraging Multilingual News Websites for Building a Kurdish Parallel Corpus

10/04/2020
by   Sina Ahmadi, et al.
0

Machine translation has been a major motivation of development in natural language processing. Despite the burgeoning achievements in creating more efficient machine translation systems thanks to deep learning methods, parallel corpora have remained indispensable for progress in the field. In an attempt to create parallel corpora for the Kurdish language, in this paper, we describe our approach in retrieving potentially-alignable news articles from multi-language websites and manually align them across dialects and languages based on lexical similarity and transliteration of scripts. We present a corpus containing 12,327 translation pairs in the two major dialects of Kurdish, Sorani and Kurmanji. We also provide 1,797 and 650 translation pairs in English-Kurmanji and English-Sorani. The corpus is publicly available under the CC BY-NC-SA 4.0 license.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/07/2018

MIZAN: A Large Persian-English Parallel Corpus

One of the most major and essential tasks in natural language processing...
01/24/2019

Automatic Parallel Corpus Creation for Hindi-English News Translation Task

The parallel corpus for multilingual NLP tasks, deep learning applicatio...
01/27/2020

PMIndia – A Collection of Parallel Corpora of Languages of India

Parallel text is required for building high-quality machine translation ...
06/17/2021

Central Kurdish machine translation: First large scale parallel corpus and experiments

While the computational processing of Kurdish has experienced a relative...
08/05/2020

Designing the Business Conversation Corpus

While the progress of machine translation of written text has come far i...
07/14/2021

ParCourE: A Parallel Corpus Explorer for a Massively Multilingual Corpus

With more than 7000 languages worldwide, multilingual natural language p...
05/21/2020

MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora

Multi-word expressions (MWEs) are a hot topic in research in natural lan...

Code Repositories

InterdialectCorpus

A parallel corpus of Sorani, Kurmanji and English


view repo