Boosting the Certified Robustness of L-infinity Distance Nets

10/13/2021
by   Bohang Zhang, et al.
0

Recently, Zhang et al. (2021) developed a new neural network architecture based on ℓ_∞-distance functions, which naturally possesses certified robustness by its construction. Despite the excellent theoretical properties, the model so far can only achieve comparable performance to conventional networks. In this paper, we significantly boost the certified robustness of ℓ_∞-distance nets through a careful analysis of its training process. In particular, we show the ℓ_p-relaxation, a crucial way to overcome the non-smoothness of the model, leads to an unexpected large Lipschitz constant at the early training stage. This makes the optimization insufficient using hinge loss and produces sub-optimal solutions. Given these findings, we propose a simple approach to address the issues above by using a novel objective function that combines a scaled cross-entropy loss with clipped hinge loss. Our experiments show that using the proposed training strategy, the certified accuracy of ℓ_∞-distance net can be dramatically improved from 33.30 to 40.06 other approaches in this area. Such a result clearly demonstrates the effectiveness and potential of ℓ_∞-distance net for certified robustness.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/20/2020

Towards adversarial robustness with 01 loss neural networks

Motivated by the general robustness properties of the 01 loss we propose...
research
05/10/2021

Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels

We propose two novel loss functions based on Jensen-Shannon divergence f...
research
07/01/2021

Boosting Certified ℓ_∞ Robustness with EMA Method and Ensemble Model

The neural network with 1-Lipschitz property based on ℓ_∞-dist neuron ha...
research
10/04/2022

Rethinking Lipschitz Neural Networks for Certified L-infinity Robustness

Designing neural networks with bounded Lipschitz constant is a promising...
research
10/26/2021

Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias

The generalization mystery of overparametrized deep nets has motivated e...
research
02/24/2020

Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inference

Deep networks were recently suggested to face the odds between accuracy ...
research
05/30/2021

Parameter Estimation for the SEIR Model Using Recurrent Nets

The standard way to estimate the parameters Θ_SEIR (e.g., the transmissi...

Please sign up or login with your details

Forgot password? Click here to reset