Imbalanced Adversarial Training with Reweighting

07/28/2021
by   Wentao Wang, et al.
0

Adversarial training has been empirically proven to be one of the most effective and reliable defense methods against adversarial attacks. However, almost all existing studies about adversarial training are focused on balanced datasets, where each class has an equal amount of training examples. Research on adversarial training with imbalanced training datasets is rather limited. As the initial effort to investigate this problem, we reveal the facts that adversarially trained models present two distinguished behaviors from naturally trained models in imbalanced datasets: (1) Compared to natural training, adversarially trained models can suffer much worse performance on under-represented classes, when the training dataset is extremely imbalanced. (2) Traditional reweighting strategies may lose efficacy to deal with the imbalance issue for adversarial training. For example, upweighting the under-represented classes will drastically hurt the model's performance on well-represented classes, and as a result, finding an optimal reweighting value can be tremendously challenging. In this paper, to further understand our observations, we theoretically show that the poor data separability is one key reason causing this strong tension between under-represented and well-represented classes. Motivated by this finding, we propose Separable Reweighted Adversarial Training (SRAT) to facilitate adversarial training under imbalanced scenarios, by learning more separable features for different classes. Extensive experiments on various datasets verify the effectiveness of the proposed framework.

READ FULL TEXT

page 9

page 14

research
10/26/2022

Improving Adversarial Robustness with Self-Paced Hard-Class Pair Reweighting

Deep Neural Networks are vulnerable to adversarial attacks. Among many d...
research
09/12/2019

Transferable Adversarial Robustness using Adversarially Trained Autoencoders

Machine learning has proven to be an extremely useful tool for solving c...
research
03/30/2021

Class-Aware Robust Adversarial Training for Object Detection

Object detection is an important computer vision task with plenty of rea...
research
02/20/2020

Boosting Adversarial Training with Hypersphere Embedding

Adversarial training (AT) is one of the most effective defenses to impro...
research
07/14/2023

Omnipotent Adversarial Training for Unknown Label-noisy and Imbalanced Datasets

Adversarial training is an important topic in robust deep learning, but ...
research
03/17/2022

On the Properties of Adversarially-Trained CNNs

Adversarial Training has proved to be an effective training paradigm to ...
research
03/12/2022

Towards Equal Opportunity Fairness through Adversarial Learning

Adversarial training is a common approach for bias mitigation in natural...

Please sign up or login with your details

Forgot password? Click here to reset