Provable Robust Classification via Learned Smoothed Densities

05/09/2020
by   Saeed Saremi, et al.
7

Smoothing classifiers and probability density functions with Gaussian kernels appear unrelated, but in this work, they are unified for the problem of robust classification. The key building block is approximating the energy function of the random variable Y=X+N(0,σ^2 I_d) with a neural network which we use to formulate the problem of robust classification in terms of x(Y), the Bayes estimator of X given the noisy measurements Y. We introduce empirical Bayes smoothed classifiers within the framework of randomized smoothing and study it theoretically for the two-class linear classifier, where we show one can improve their robustness above the margin. We test the theory on MNIST and we show that with a learned smoothed energy function and a linear classifier we can achieve provable ℓ_2 robust accuracies that are competitive with empirical defenses. This setup can be significantly improved by learning empirical Bayes smoothed classifiers with adversarial training and on MNIST we show that we can achieve provable robust accuracies higher than the state-of-the-art empirical defenses in a range of radii. We discuss some fundamental challenges of randomized smoothing based on a geometric interpretation due to concentration of Gaussians in high dimensions, and we finish the paper with a proposal for using walk-jump sampling, itself based on learned smoothed densities, for robust classification.

READ FULL TEXT

page 5

page 12

page 20

research
06/09/2019

Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers

Recent works have shown the effectiveness of randomized smoothing as a s...
research
07/29/2020

Unnormalized Variational Bayes

We unify empirical Bayes and variational Bayes for approximating unnorma...
research
01/08/2020

MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius

Adversarial training is one of the most popular ways to learn robust mod...
research
02/07/2020

Certified Robustness to Label-Flipping Attacks via Randomized Smoothing

Machine learning algorithms are known to be susceptible to data poisonin...
research
03/06/2019

Neural Empirical Bayes

We formulate a novel framework that unifies kernel density estimation an...
research
12/18/2021

Multimeasurement Generative Models

We formally map the problem of sampling from an unknown distribution wit...
research
09/18/2020

A Framework of Randomized Selection Based Certified Defenses Against Data Poisoning Attacks

Neural network classifiers are vulnerable to data poisoning attacks, as ...

Please sign up or login with your details

Forgot password? Click here to reset