Log In Sign Up

Bias Challenges in Counterfactual Data Augmentation

by   S Chandra Mouli, et al.

Deep learning models tend not to be out-of-distribution robust primarily due to their reliance on spurious features to solve the task. Counterfactual data augmentations provide a general way of (approximately) achieving representations that are counterfactual-invariant to spurious features, a requirement for out-of-distribution (OOD) robustness. In this work, we show that counterfactual data augmentations may not achieve the desired counterfactual-invariance if the augmentation is performed by a context-guessing machine, an abstract machine that guesses the most-likely context of a given input. We theoretically analyze the invariance imposed by such counterfactual data augmentations and describe an exemplar NLP task where counterfactual data augmentation by a context-guessing machine does not lead to robust OOD classifiers.


page 1

page 2

page 3

page 4


Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests

Informally, a `spurious correlation' is the dependence of a model on som...

SpanDrop: Simple and Effective Counterfactual Learning for Long Sequences

Distilling supervision signal from a long sequence to make predictions i...

Gradient-based Counterfactual Explanations using Tractable Probabilistic Models

Counterfactual examples are an appealing class of post-hoc explanations ...

In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?

It is often said that a deep learning model is "invariant" to some speci...

Data-Augmented Counterfactual Learning for Bundle Recommendation

Bundle Recommendation (BR) aims at recommending bundled items on online ...

Counterfactual Plans under Distributional Ambiguity

Counterfactual explanations are attracting significant attention due to ...

Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text

Machine Learning has seen tremendous growth recently, which has led to a...