Balancing Robustness and Sensitivity using Feature Contrastive Learning

by   Seungyeon Kim, et al.

It is generally believed that robust training of extremely large networks is critical to their success in real-world applications. However, when taken to the extreme, methods that promote robustness can hurt the model's sensitivity to rare or underrepresented patterns. In this paper, we discuss this trade-off between sensitivity and robustness to natural (non-adversarial) perturbations by introducing two notions: contextual feature utility and contextual feature sensitivity. We propose Feature Contrastive Learning (FCL) that encourages a model to be more sensitive to the features that have higher contextual utility. Empirical results demonstrate that models trained with FCL achieve a better balance of robustness and sensitivity, leading to improved generalization in the presence of noise on both vision and NLP datasets.


Causally Estimating the Sensitivity of Neural NLP Models to Spurious Features

Recent work finds modern natural language processing (NLP) models relyin...

Towards Adversarial Robustness of Deep Vision Algorithms

Deep learning methods have achieved great success in solving computer vi...

Investigating Vulnerabilities of Deep Neural Policies

Reinforcement learning policies based on deep neural networks are vulner...

Pareto Adversarial Robustness: Balancing Spatial Robustness and Sensitivity-based Robustness

Adversarial robustness, which mainly contains sensitivity-based robustne...

Cost-Sensitive Robustness against Adversarial Examples

Several recent works have developed methods for training classifiers tha...

There Is No Free Lunch In Adversarial Robustness (But There Are Unexpected Benefits)

We provide a new understanding of the fundamental nature of adversariall...

Please sign up or login with your details

Forgot password? Click here to reset