Enhancing Adversarial Training via Reweighting Optimization Trajectory

by   Tianjin Huang, et al.
TU Eindhoven

Despite the fact that adversarial training has become the de facto method for improving the robustness of deep neural networks, it is well-known that vanilla adversarial training suffers from daunting robust overfitting, resulting in unsatisfactory robust generalization. A number of approaches have been proposed to address these drawbacks such as extra regularization, adversarial weights perturbation, and training with more data over the last few years. However, the robust generalization improvement is yet far from satisfactory. In this paper, we approach this challenge with a brand new perspective – refining historical optimization trajectories. We propose a new method named Weighted Optimization Trajectories (WOT) that leverages the optimization trajectories of adversarial training in time. We have conducted extensive experiments to demonstrate the effectiveness of WOT under various state-of-the-art adversarial attacks. Our results show that WOT integrates seamlessly with the existing adversarial training methods and consistently overcomes the robust overfitting issue, resulting in better adversarial robustness. For example, WOT boosts the robust accuracy of AT-PGD under AA-L_∞ attack by 1.53% ∼ 6.11% and meanwhile increases the clean accuracy by 0.55%∼5.47% across SVHN, CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets.


page 1

page 2

page 3

page 4


CAT: Customized Adversarial Training for Improved Robustness

Adversarial training has become one of the most effective methods for im...

Adversarial Feature Stacking for Accurate and Robust Predictions

Deep Neural Networks (DNNs) have achieved remarkable performance on a va...

Adversarial Training with Stochastic Weight Average

Adversarial training deep neural networks often experience serious overf...

Enhancing Adversarial Training with Feature Separability

Deep Neural Network (DNN) are vulnerable to adversarial attacks. As a co...

Stability Analysis and Generalization Bounds of Adversarial Training

In adversarial machine learning, deep neural networks can fit the advers...

OTJR: Optimal Transport Meets Optimal Jacobian Regularization for Adversarial Robustness

Deep neural networks are widely recognized as being vulnerable to advers...

Annealing Self-Distillation Rectification Improves Adversarial Training

In standard adversarial training, models are optimized to fit one-hot la...

Please sign up or login with your details

Forgot password? Click here to reset