Log In Sign Up

Adversarial Robustness via Fisher-Rao Regularization

by   Marine Picot, et al.

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce FIRE, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, for a simple linear and Gaussian model, we show that all Pareto-optimal points in the accuracy-robustness region can be reached by FIRE while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to 2% of improvements in terms of robustness while reducing the training time by 20% over the best-performing methods.


page 3

page 4


Towards adversarial robustness with 01 loss neural networks

Motivated by the general robustness properties of the 01 loss we propose...

Achieving robustness in classification using optimal transport with hinge regularization

We propose a new framework for robust binary classification, with Deep N...

Constraining Logits by Bounded Function for Adversarial Robustness

We propose a method for improving adversarial robustness by addition of ...

On Certifying Robust Models by Polyhedral Envelope

Certifying neural networks enables one to offer guarantees on a model's ...

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

Previous work shows that adversarially robust generalization requires la...

Measuring Data Leakage in Machine-Learning Models with Fisher Information

Machine-learning models contain information about the data they were tra...

von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning

Recent work has argued that classification losses utilizing softmax cros...