Avoiding spurious correlations via logit correction

12/02/2022
by   Sheng Liu, et al.
10

Empirical studies suggest that machine learning models trained with empirical risk minimization (ERM) often rely on attributes that may be spuriously correlated with the class labels. Such models typically lead to poor performance during inference for data lacking such correlations. In this work, we explicitly consider a situation where potential spurious correlations are present in the majority of training data. In contrast with existing approaches, which use the ERM model outputs to detect the samples without spurious correlations, and either heuristically upweighting or upsampling those samples; we propose the logit correction (LC) loss, a simple yet effective improvement on the softmax cross-entropy loss, to correct the sample logit. We demonstrate that minimizing the LC loss is equivalent to maximizing the group-balanced accuracy, so the proposed LC could mitigate the negative impacts of spurious correlations. Our extensive experimental results further reveal that the proposed LC loss outperforms the SoTA solutions on multiple popular benchmarks by a large margin, an average 5.5 spurious attribute labels. LC is also competitive with oracle methods that make use of the attribute labels. Code is available at https://github.com/shengliu66/LC.

READ FULL TEXT
research
03/03/2022

Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlations

Spurious correlations pose a major challenge for robust machine learning...
research
02/11/2023

Jaccard Metric Losses: Optimizing the Jaccard Index with Soft Labels

IoU losses are surrogates that directly optimize the Jaccard index. In s...
research
09/30/2022

MaskTune: Mitigating Spurious Correlations by Forcing to Explore

A fundamental challenge of over-parameterized deep learning models is le...
research
03/22/2023

Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models

State-of-the-art machine learning models often learn spurious correlatio...
research
09/03/2022

Noise-Robust Bidirectional Learning with Dynamic Sample Reweighting

Deep neural networks trained with standard cross-entropy loss are more p...
research
05/26/2021

Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers

We propose Predict then Interpolate (PI), a simple algorithm for learnin...
research
06/21/2023

Towards Mitigating Spurious Correlations in the Wild: A Benchmark a more Realistic Dataset

Deep neural networks often exploit non-predictive features that are spur...

Please sign up or login with your details

Forgot password? Click here to reset