Improving Corruption and Adversarial Robustness by Enhancing Weak Subnets

01/30/2022
by   Yong Guo, et al.
0

Deep neural networks have achieved great success in many computer vision tasks. However, deep networks have been shown to be very susceptible to corrupted or adversarial images, which often result in significant performance drops. In this paper, we observe that weak subnetwork (subnet) performance is correlated with a lack of robustness against corruptions and adversarial attacks. Based on that observation, we propose a novel robust training method which explicitly identifies and enhances weak subnets (EWS) during training to improve robustness. Specifically, we develop a search algorithm to find particularly weak subnets and propose to explicitly strengthen them via knowledge distillation from the full network. We show that our EWS greatly improves the robustness against corrupted images as well as the accuracy on clean data. Being complementary to many state-of-the-art data augmentation approaches, EWS consistently improves corruption robustness on top of many of these approaches. Moreover, EWS is also able to boost the adversarial robustness when combined with popular adversarial training methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2019

Adversarially Robust Distillation

Knowledge distillation is effective for producing small high-performance...
research
03/01/2021

A Multiclass Boosting Framework for Achieving Fast and Provable Adversarial Robustness

Alongside the well-publicized accomplishments of deep neural networks th...
research
09/11/2019

Structural Robustness for Deep Learning Architectures

Deep Networks have been shown to provide state-of-the-art performance in...
research
06/06/2023

Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings

Research on adversarial robustness is primarily focused on image and tex...
research
01/05/2022

Corrupting Data to Remove Deceptive Perturbation: Using Preprocessing Method to Improve System Robustness

Although deep neural networks have achieved great performance on classif...
research
03/15/2023

Improving Adversarial Robustness with Hypersphere Embedding and Angular-based Regularizations

Adversarial training (AT) methods have been found to be effective agains...
research
05/14/2023

Improving Defensive Distillation using Teacher Assistant

Adversarial attacks pose a significant threat to the security and safety...

Please sign up or login with your details

Forgot password? Click here to reset