Towards Robust Deep Neural Networks

10/27/2018
by   Timothy E. Wang, et al.
0

We examine the relationship between the energy landscape of neural networks and their robustness to adversarial attacks. Combining energy landscape techniques developed in computational chemistry with tools drawn from formal methods, we produce empirical evidence that networks corresponding to lower-lying minima in the landscape tend to be more robust. The robustness measure used is the inverse of the sensitivity measure, which we define as the volume of an over-approximation of the reachable set of network outputs under all additive l_∞ bounded perturbations on the input data. We present a novel loss function which contains a weighted sensitivity component in addition to the traditional task-oriented and regularization terms. In our experiments on standard machine learning and computer vision datasets (e.g., Iris and MNIST), we show that the proposed loss function leads to networks which reliably optimize the robustness measure as well as other related metrics of adversarial robustness without significant degradation in the classification error.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/27/2020

Adversarially Robust Learning via Entropic Regularization

In this paper we propose a new family of algorithms for training adversa...
research
10/26/2021

Adversarial Robustness in Multi-Task Learning: Promises and Illusions

Vulnerability to adversarial attacks is a well-known weakness of Deep Ne...
research
03/03/2021

Formalizing Generalization and Robustness of Neural Networks to Weight Perturbations

Studying the sensitivity of weight perturbation in neural networks and i...
research
05/27/2022

Standalone Neural ODEs with Sensitivity Analysis

This paper presents the Standalone Neural ODE (sNODE), a continuous-dept...
research
04/06/2018

The Loss Surface of XOR Artificial Neural Networks

Training an artificial neural network involves an optimization process o...
research
04/17/2018

Learning how to be robust: Deep polynomial regression

Polynomial regression is a recurrent problem with a large number of appl...
research
03/04/2020

The Impact of Hole Geometry on Relative Robustness of In-Painting Networks: An Empirical Study

In-painting networks use existing pixels to generate appropriate pixels ...

Please sign up or login with your details

Forgot password? Click here to reset