On the Unintended Social Bias of Training Language Generation Models with Data from Local Media

11/01/2019
by   Omar U. Florez, et al.
0

There are concerns that neural language models may preserve some of the stereotypes of the underlying societies that generate the large corpora needed to train these models. For example, gender bias is a significant problem when generating text, and its unintended memorization could impact the user experience of many applications (e.g., the smart-compose feature in Gmail). In this paper, we introduce a novel architecture that decouples the representation learning of a neural model from its memory management role. This architecture allows us to update a memory module with an equal ratio across gender types addressing biased correlations directly in the latent space. We experimentally show that our approach can mitigate the gender bias amplification in the automatic generation of articles news while providing similar perplexity values when extending the Sequence2Sequence architecture.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2022

Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information

The representations in large language models contain multiple types of g...
research
09/18/2023

Bias of AI-Generated Content: An Examination of News Produced by Large Language Models

Large language models (LLMs) have the potential to transform our lives a...
research
07/18/2022

Selection Bias Induced Spurious Correlations in Large Language Models

In this work we show how large language models (LLMs) can learn statisti...
research
10/05/2020

Viable Threat on News Reading: Generating Biased News Using Natural Language Models

Recent advancements in natural language generation has raised serious co...
research
05/29/2023

Self Information Update for Large Language Models through Mitigating Exposure Bias

Current LLMs have demonstrated remarkable capabilities in addressing use...
research
11/21/2022

Validating Large Language Models with ReLM

Although large language models (LLMs) have been touted for their ability...
research
01/21/2023

Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models

Groundbreaking inventions and highly significant performance improvement...

Please sign up or login with your details

Forgot password? Click here to reset