DeepAI AI Chat
Log In Sign Up

Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models

10/01/2021
by   Robert Wolfe, et al.
0

We use a dataset of U.S. first names with labels based on predominant gender and racial group to examine the effect of training corpus frequency on tokenization, contextualization, similarity to initial representation, and bias in BERT, GPT-2, T5, and XLNet. We show that predominantly female and non-white names are less frequent in the training corpora of these four language models. We find that infrequent names are more self-similar across contexts, with Spearman's r between frequency and self-similarity as low as -.763. Infrequent names are also less similar to initial representation, with Spearman's r between frequency and linear centered kernel alignment (CKA) similarity to initial representation as high as .702. Moreover, we find Spearman's r between racial bias and name frequency in BERT of .492, indicating that lower-frequency minority group names are more associated with unpleasantness. Representations of infrequent names undergo more processing, but are more self-similar, indicating that models rely on less context-informed representations of uncommon and minority names which are overfit to a lower number of observed contexts.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/06/2020

"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Pre-trained language models (LMs) may perpetuate biases originating in t...
12/05/2021

VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning

Variable names are critical for conveying intended program behavior. Mac...
01/23/2019

Attenuating Bias in Word Vectors

Word vector representations are well developed tools for various NLP and...
06/21/2018

An empirical study on the names of points of interest and their changes with geographic distance

While Points Of Interest (POIs), such as restaurants, hotels, and barber...
08/08/2022

Debiased Large Language Models Still Associate Muslims with Uniquely Violent Acts

Recent work demonstrates a bias in the GPT-3 model towards generating vi...
09/29/2022

Temporal Analysis and Gender Bias in Computing

Recent studies of gender bias in computing use large datasets involving ...
05/10/2022

Richer Countries and Richer Representations

We examine whether some countries are more richly represented in embeddi...