Architecture Selection via the Trade-off Between Accuracy and Robustness

06/04/2019
by   Zhun Deng, et al.
0

We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we introduce a simple trade-off curve, define and study an influence function that captures the sensitivity, under adversarial attack, of the optima of a given loss function. We further show how adversarial training regularizes the parameters in an over-parameterized linear model, recovering the LASSO and ridge regression as special cases, which also allows us to theoretically analyze the behavior of the trade-off curve. In experiments, we demonstrate the corresponding trade-off curves of neural networks and how they vary with respect to factors such as number of layers, neurons, and across different network structures. Such information provides a useful guideline to architecture selection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/20/2022

Learning Sample Reweighting for Accuracy and Adversarial Robustness

There has been great interest in enhancing the robustness of neural netw...
research
02/26/2020

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

The classical bias-variance trade-off predicts that bias decreases and v...
research
08/16/2020

Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks

Adversarial training has been proven to be an effective technique for im...
research
08/15/2021

Deep Adversarially-Enhanced k-Nearest Neighbors

Recent works have theoretically and empirically shown that deep neural n...
research
08/16/2021

Neural Architecture Dilation for Adversarial Robustness

With the tremendous advances in the architecture and scale of convolutio...
research
02/21/2022

Robustness and Accuracy Could Be Reconcilable by (Proper) Definition

The trade-off between robustness and accuracy has been widely studied in...

Please sign up or login with your details

Forgot password? Click here to reset