Adaptive Verifiable Training Using Pairwise Class Similarity

by   Shiqi Wang, et al.

Verifiable training has shown success in creating neural networks that are provably robust to a given amount of noise. However, despite only enforcing a single robustness criterion, its performance scales poorly with dataset complexity. On CIFAR10, a non-robust LeNet model has a 21.63 a model created using verifiable training and a L-infinity robustness criterion of 8/255, has an error rate of 57.10 labeling visually similar classes, the model's error rate is as high as 61.65 We attribute the loss in performance to inter-class similarity. Similar classes (i.e., close in the feature space) increase the difficulty of learning a robust model. While it's desirable to train a robust model for a large robustness region, pairwise class similarities limit the potential gains. Also, consideration must be made regarding the relative cost of mistaking similar classes. In security or safety critical tasks, similar classes are likely to belong to the same group, and thus are equally sensitive. In this work, we propose a new approach that utilizes inter-class similarity to improve the performance of verifiable training and create robust models with respect to multiple adversarial criteria. First, we use agglomerate clustering to group similar classes and assign robustness criteria based on the similarity between clusters. Next, we propose two methods to apply our approach: (1) Inter-Group Robustness Prioritization, which uses a custom loss term to create a single model with multiple robustness guarantees and (2) neural decision trees, which trains multiple sub-classifiers with different robustness guarantees and combines them in a decision tree architecture. On Fashion-MNIST and CIFAR10, our approach improves clean performance by 9.63 respectively. On CIFAR100, our approach improves clean performance by 26.32



There are no comments yet.


page 1

page 2

page 3

page 4


Enforcing robust control guarantees within neural network policies

When designing controllers for safety-critical systems, practitioners of...

Provable Robustness of ReLU networks via Maximization of Linear Regions

It has been shown that neural network classifiers are not robust. This r...

Exploiting Class Similarity for Machine Learning with Confidence Labels and Projective Loss Functions

Class labels used for machine learning are relatable to each other, with...

Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness

Previous work shows that adversarially robust generalization requires la...

Cost-Sensitive Robustness against Adversarial Examples

Several recent works have developed methods for training classifiers tha...

On Single Source Robustness in Deep Fusion Models

Algorithms that fuse multiple input sources benefit from both complement...

Detecting and Understanding Real-World Differential Performance Bugs in Machine Learning Libraries

Programming errors that degrade the performance of systems are widesprea...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.