Measuring and Reducing Gendered Correlations in Pre-trained Models

10/12/2020
by   Kellie Webster, et al.
1

Pre-trained models have revolutionized natural language understanding. However, researchers have found they can encode artifacts undesired in many applications, such as professions correlating with one gender more than another. We explore such gendered correlations as a case study for how to address unintended correlations in pre-trained models. We define metrics and reveal that it is possible for models with similar accuracy to encode correlations at very different rates. We show how measured correlations can be reduced with general-purpose techniques, and highlight the trade offs different strategies have. With these results, we make recommendations for training robust models: (1) carefully evaluate unintended correlations, (2) be mindful of seemingly innocuous configuration differences, and (3) focus on general mitigations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2023

An Empirical Comparison of Pre-Trained Models of Source Code

While a large number of pre-trained models of source code have been succ...
research
07/14/2020

An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models

Recent work has shown that pre-trained language models such as BERT impr...
research
06/04/2018

Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark

The deep learning community has proposed optimizations spanning hardware...
research
06/17/2022

How robust are pre-trained models to distribution shift?

The vulnerability of machine learning models to spurious correlations ha...
research
10/03/2021

Adversarial Examples Generation for Reducing Implicit Gender Bias in Pre-trained Models

Over the last few years, Contextualized Pre-trained Neural Language Mode...
research
08/30/2023

CorrEmbed: Evaluating Pre-trained Model Image Similarity Efficacy with a Novel Metric

Detecting visually similar images is a particularly useful attribute to ...
research
07/09/2021

Can Deep Neural Networks Predict Data Correlations from Column Names?

For humans, it is often possible to predict data correlations from colum...

Please sign up or login with your details

Forgot password? Click here to reset