Adaptive Verifiable Training Using Pairwise Class Similarity

12/14/2020
by   Shiqi Wang, et al.
0

Verifiable training has shown success in creating neural networks that are provably robust to a given amount of noise. However, despite only enforcing a single robustness criterion, its performance scales poorly with dataset complexity. On CIFAR10, a non-robust LeNet model has a 21.63 a model created using verifiable training and a L-infinity robustness criterion of 8/255, has an error rate of 57.10 labeling visually similar classes, the model's error rate is as high as 61.65 We attribute the loss in performance to inter-class similarity. Similar classes (i.e., close in the feature space) increase the difficulty of learning a robust model. While it's desirable to train a robust model for a large robustness region, pairwise class similarities limit the potential gains. Also, consideration must be made regarding the relative cost of mistaking similar classes. In security or safety critical tasks, similar classes are likely to belong to the same group, and thus are equally sensitive. In this work, we propose a new approach that utilizes inter-class similarity to improve the performance of verifiable training and create robust models with respect to multiple adversarial criteria. First, we use agglomerate clustering to group similar classes and assign robustness criteria based on the similarity between clusters. Next, we propose two methods to apply our approach: (1) Inter-Group Robustness Prioritization, which uses a custom loss term to create a single model with multiple robustness guarantees and (2) neural decision trees, which trains multiple sub-classifiers with different robustness guarantees and combines them in a decision tree architecture. On Fashion-MNIST and CIFAR10, our approach improves clean performance by 9.63 respectively. On CIFAR100, our approach improves clean performance by 26.32

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/16/2020

Enforcing robust control guarantees within neural network policies

When designing controllers for safety-critical systems, practitioners of...
10/17/2018

Provable Robustness of ReLU networks via Maximization of Linear Regions

It has been shown that neural network classifiers are not robust. This r...
03/25/2021

Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions

Class labels used for machine learning are relatable to each other, with...
05/25/2019

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

Previous work shows that adversarially robust generalization requires la...
10/22/2018

Cost-Sensitive Robustness against Adversarial Examples

Several recent works have developed methods for training classifiers tha...
06/11/2019

On Single Source Robustness in Deep Fusion Models

Algorithms that fuse multiple input sources benefit from both complement...
06/03/2020

Detecting and Understanding Real-World Differential Performance Bugs in Machine Learning Libraries

Programming errors that degrade the performance of systems are widesprea...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.