Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization

08/11/2021
by   Wei Zhu, et al.
0

Deep learning algorithms mine knowledge from the training data and thus would likely inherit the dataset's bias information. As a result, the obtained model would generalize poorly and even mislead the decision process in real-life applications. We propose to remove the bias information misused by the target task with a cross-sample adversarial debiasing (CSAD) method. CSAD explicitly extracts target and bias features disentangled from the latent representation generated by a feature extractor and then learns to discover and remove the correlation between the target and bias features. The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator. Moreover, we propose joint content and local structural representation learning to boost mutual information estimation for better performance. We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2023

Conditional Mutual Information for Disentangled Representations in Reinforcement Learning

Reinforcement Learning (RL) environments can produce training data with ...
research
04/26/2023

Understanding the limitation of Total Correlation Estimation Based on Mutual Information Bounds

The total correlation(TC) is a crucial index to measure the correlation ...
research
02/21/2023

Scalable Infomin Learning

The task of infomin learning aims to learn a representation with high ut...
research
12/26/2018

Learning Not to Learn: Training Deep Neural Networks with Biased Data

We propose a novel regularization algorithm to train deep neural network...
research
10/26/2022

Disentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustness

Adversarial vulnerability remains a major obstacle to constructing relia...
research
05/03/2020

Mutual Information Gradient Estimation for Representation Learning

Mutual Information (MI) plays an important role in representation learni...
research
10/31/2022

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric...

Please sign up or login with your details

Forgot password? Click here to reset