Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation

12/02/2021
by   Yeonsung Jung, et al.
0

Despite their remarkable ability to generalize with over-capacity networks, deep neural networks often learn to abuse spurious biases in the data instead of using the actual task-related information. Since such shortcuts are only effective within the collected dataset, the resulting biased model underperforms on real-world inputs, or cause unintended social repercussions such as gender discrimination. To counteract the influence of bias, existing methods either exploit auxiliary information which is rarely obtainable in practice, or sift for bias-free samples in the training data, hoping for the sufficient existence of clean samples. However, such presumptions about the data are not always guaranteed. In this paper, we propose Contrastive Debiasing via Generative Bias-transformation (CDvG) which is capable of operating in more general environments where existing methods break down due to unmet presumptions such as insufficient bias-free samples. Motivated by our observation that not only discriminative models, as previously known, but also generative models tend to focus on the bias when possible, CDvG uses a translation model to transform the bias in the sample to another mode of bias while preserving task-relevant information. Through contrastive learning, we set transformed biased views against another, learning bias-invariant representations. Experimental results on synthetic and real-world datasets demonstrate that our framework outperforms the current state-of-the-arts, and effectively prevents the models from being biased even when bias-free samples are extremely scarce.

READ FULL TEXT

page 3

page 4

page 13

research
05/06/2023

Echoes: Unsupervised Debiasing via Pseudo-bias Labeling in an Echo Chamber

Neural networks often learn spurious correlations when exposed to biased...
research
05/29/2022

BiasEnsemble: Revisiting the Importance of Amplifying Bias for Debiasing

In image classification, "debiasing" aims to train a classifier to be le...
research
12/11/2022

Feature-Level Debiased Natural Language Understanding

Natural language understanding (NLU) models often rely on dataset biases...
research
10/11/2022

Efficient debiasing with contrastive weight pruning

Neural networks are often biased to spuriously correlated features that ...
research
11/04/2022

SelecMix: Debiased Learning by Contradicting-pair Sampling

Neural networks trained with ERM (empirical risk minimization) sometimes...
research
09/09/2021

IFBiD: Inference-Free Bias Detection

This paper is the first to explore an automatic way to detect bias in de...
research
07/13/2019

The Futility of Bias-Free Learning and Search

Building on the view of machine learning as search, we demonstrate the n...

Please sign up or login with your details

Forgot password? Click here to reset