Generalization in anti-causal learning

12/03/2018
by   Niki Kilbertus, et al.
0

The ability to learn and act in novel situations is still a prerogative of animate intelligence, as current machine learning methods mostly fail when moving beyond the standard i.i.d. setting. What is the reason for this discrepancy? Most machine learning tasks are anti-causal, i.e., we infer causes (labels) from effects (observations). Typically, in supervised learning we build systems that try to directly invert causal mechanisms. Instead, in this paper we argue that strong generalization capabilities crucially hinge on searching and validating meaningful hypotheses, requiring access to a causal model. In such a framework, we want to find a cause that leads to the observed effect. Anti-causal models are used to drive this search, but a causal model is required for validation. We investigate the fundamental differences between causal and anti-causal tasks, discuss implications for topics ranging from adversarial attacks to disentangling factors of variation, and provide extensive evidence from the literature to substantiate our view. We advocate for incorporating causal models in supervised learning to shift the paradigm from inference only, to search and validation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2022

On Causality in Domain Adaptation and Semi-Supervised Learning: an Information-Theoretic Analysis

The establishment of the link between causality and unsupervised domain ...
research
06/27/2012

On Causal and Anticausal Learning

We consider the problem of function estimation in the case where an unde...
research
12/15/2015

Causal and anti-causal learning in pattern recognition for neuroimaging

Pattern recognition in neuroimaging distinguishes between two types of m...
research
09/18/2022

Membership Inference Attacks and Generalization: A Causal Perspective

Membership inference (MI) attacks highlight a privacy weakness in presen...
research
01/11/2018

Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution

Current machine learning systems operate, almost exclusively, in a stati...
research
03/24/2019

Machine Learning Methods Economists Should Know About

We discuss the relevance of the recent Machine Learning (ML) literature ...

Please sign up or login with your details

Forgot password? Click here to reset