Evaluating Bias In Dutch Word Embeddings

Recent research in Natural Language Processing has revealed that word embeddings can encode social biases present in the training data which can affect minorities in real world applications. This paper explores the gender bias implicit in Dutch embeddings while investigating whether English language based approaches can also be used in Dutch. We implement the Word Embeddings Association Test (WEAT), Clustering and Sentence Embeddings Association Test (SEAT) methods to quantify the gender bias in Dutch word embeddings, then we proceed to reduce the bias with Hard-Debias and Sent-Debias mitigation methods and finally we evaluate the performance of the debiased embeddings in downstream tasks. The results suggest that, among others, gender bias is present in traditional and contextualized Dutch word embeddings. We highlight how techniques used to measure and reduce bias created for English can be used in Dutch embeddings by adequately translating the data and taking into account the unique characteristics of the language. Furthermore, we analyze the effect of the debiasing techniques on downstream tasks which show a negligible impact on traditional embeddings and a 2 embeddings. Finally, we release the translated Dutch datasets to the public along with the traditional embeddings with mitigated bias.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/14/2021

[RE] Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation

Despite widespread use in natural language processing (NLP) tasks, word ...
research
12/05/2019

Measuring Social Bias in Knowledge Graph Embeddings

It has recently been shown that word embeddings encode social biases, wi...
research
09/28/2021

Marked Attribute Bias in Natural Language Inference

Reporting and providing test sets for harmful bias in NLP applications i...
research
06/16/2018

Biased Embeddings from Wild Data: Measuring, Understanding and Removing

Many modern Artificial Intelligence (AI) systems make use of data embedd...
research
07/16/2020

Towards Debiasing Sentence Representations

As natural language processing methods are increasingly deployed in real...
research
12/31/2020

Intrinsic Bias Metrics Do Not Correlate with Application Bias

Natural Language Processing (NLP) systems learn harmful societal biases ...
research
06/20/2020

MDR Cluster-Debias: A Nonlinear WordEmbedding Debiasing Pipeline

Existing methods for debiasing word embeddings often do so only superfic...

Please sign up or login with your details

Forgot password? Click here to reset