Learning Stable Classifiers by Transferring Unstable Features

06/15/2021
by   Yujia Bao, et al.
7

We study transfer learning in the presence of spurious correlations. We experimentally demonstrate that directly transferring the stable feature extractor learned on the source task may not eliminate these biases for the target task. However, we hypothesize that the unstable features in the source task and those in the target task are directly related. By explicitly informing the target classifier of the source task's unstable features, we can regularize the biases in the target task. Specifically, we derive a representation that encodes the unstable features by contrasting different data environments in the source task. On the target task, we cluster data from this representation, and achieve robustness by minimizing the worst-case risk across all clusters. We evaluate our method on both text and image classifications. Empirical results demonstrate that our algorithm is able to maintain robustness on the target task, outperforming the best baseline by 22.9 transfer settings. Our code is available at https://github.com/YujiaBao/Tofu.

READ FULL TEXT
research
05/26/2021

Predict then Interpolate: A Simple Algorithm to Learn Stable Classifiers

We propose Predict then Interpolate (PI), a simple algorithm for learnin...
research
07/02/2020

Learn Faster and Forget Slower via Fast and Stable Task Adaptation

Training Deep Neural Networks (DNNs) is still highly time-consuming and ...
research
07/01/2021

Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding

The ability to transfer a policy from one environment to another is a pr...
research
10/10/2015

Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain

Transferring knowledge from prior source tasks in solving a new target t...
research
03/19/2023

Partial Network Cloning

In this paper, we study a novel task that enables partial knowledge tran...
research
12/02/2021

Editing a classifier by rewriting its prediction rules

We present a methodology for modifying the behavior of a classifier by d...
research
09/03/2023

Enhancing Infrared Small Target Detection Robustness with Bi-Level Adversarial Framework

The detection of small infrared targets against blurred and cluttered ba...

Please sign up or login with your details

Forgot password? Click here to reset