DeepAI AI Chat
Log In Sign Up

Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Models

by   Robert Wolfe, et al.

We use a dataset of U.S. first names with labels based on predominant gender and racial group to examine the effect of training corpus frequency on tokenization, contextualization, similarity to initial representation, and bias in BERT, GPT-2, T5, and XLNet. We show that predominantly female and non-white names are less frequent in the training corpora of these four language models. We find that infrequent names are more self-similar across contexts, with Spearman's r between frequency and self-similarity as low as -.763. Infrequent names are also less similar to initial representation, with Spearman's r between frequency and linear centered kernel alignment (CKA) similarity to initial representation as high as .702. Moreover, we find Spearman's r between racial bias and name frequency in BERT of .492, indicating that lower-frequency minority group names are more associated with unpleasantness. Representations of infrequent names undergo more processing, but are more self-similar, indicating that models rely on less context-informed representations of uncommon and minority names which are overfit to a lower number of observed contexts.


page 1

page 2

page 3

page 4


"You are grounded!": Latent Name Artifacts in Pre-trained Language Models

Pre-trained language models (LMs) may perpetuate biases originating in t...

VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning

Variable names are critical for conveying intended program behavior. Mac...

Attenuating Bias in Word Vectors

Word vector representations are well developed tools for various NLP and...

An empirical study on the names of points of interest and their changes with geographic distance

While Points Of Interest (POIs), such as restaurants, hotels, and barber...

Debiased Large Language Models Still Associate Muslims with Uniquely Violent Acts

Recent work demonstrates a bias in the GPT-3 model towards generating vi...

Temporal Analysis and Gender Bias in Computing

Recent studies of gender bias in computing use large datasets involving ...

Richer Countries and Richer Representations

We examine whether some countries are more richly represented in embeddi...