FreCDo: A Large Corpus for French Cross-Domain Dialect Identification

12/15/2022
by   Mihaela Gaman, et al.
0

We present a novel corpus for French dialect identification comprising 413,522 French text samples collected from public news websites in Belgium, Canada, France and Switzerland. To ensure an accurate estimation of the dialect identification performance of models, we designed the corpus to eliminate potential biases related to topic, writing style, and publication source. More precisely, the training, validation and test splits are collected from different news websites, while searching for different keywords (topics). This leads to a French cross-domain (FreCDo) dialect identification task. We conduct experiments with four competitive baselines, a fine-tuned CamemBERT model, an XGBoost based on fine-tuned CamemBERT features, a Support Vector Machines (SVM) classifier based on fine-tuned CamemBERT features, and an SVM based on word n-grams. Aside from presenting quantitative results, we also make an analysis of the most discriminative features learned by CamemBERT. Our corpus is available at https://github.com/MihaelaGaman/FreCDo.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/28/2022

Cross-domain Cross-architecture Black-box Attacks on Fine-tuned Models with Transferred Evolutionary Strategies

Fine-tuning can be vulnerable to adversarial attacks. Existing works abo...
research
01/19/2019

MOROCO: The Moldavian and Romanian Dialectal Corpus

In this work, we introduce the MOldavian and ROmanian Dialectal COrpus (...
research
08/24/2023

Financial News Analytics Using Fine-Tuned Llama 2 GPT Model

The paper considers the possibility to fine-tune Llama 2 GPT large langu...
research
12/17/2020

MASKER: Masked Keyword Regularization for Reliable Text Classification

Pre-trained language models have achieved state-of-the-art accuracies on...
research
11/04/2022

Logits are predictive of network type

We show that it is possible to predict which deep network has generated ...
research
09/24/2022

Dead or Murdered? Predicting Responsibility Perception in Femicide News Reports

Different linguistic expressions can conceptualize the same event from d...
research
04/10/2021

FreSaDa: A French Satire Data Set for Cross-Domain Satire Detection

In this paper, we introduce FreSaDa, a French Satire Data Set, which is ...

Please sign up or login with your details

Forgot password? Click here to reset