Dual Head Adversarial Training

04/21/2021
by   Yujing Jiang, et al.
0

Deep neural networks (DNNs) are known to be vulnerable to adversarial examples/attacks, raising concerns about their reliability in safety-critical applications. A number of defense methods have been proposed to train robust DNNs resistant to adversarial attacks, among which adversarial training has so far demonstrated the most promising results. However, recent studies have shown that there exists an inherent tradeoff between accuracy and robustness in adversarially-trained DNNs. In this paper, we propose a novel technique Dual Head Adversarial Training (DH-AT) to further improve the robustness of existing adversarial training methods. Different from existing improved variants of adversarial training, DH-AT modifies both the architecture of the network and the training strategy to seek more robustness. Specifically, DH-AT first attaches a second network head (or branch) to one intermediate layer of the network, then uses a lightweight convolutional neural network (CNN) to aggregate the outputs of the two heads. The training strategy is also adapted to reflect the relative importance of the two heads. We empirically show, on multiple benchmark datasets, that DH-AT can bring notable robustness improvements to existing adversarial training methods. Compared with TRADES, one state-of-the-art adversarial training method, our DH-AT can improve the robustness by 3.4 the clean accuracy by 1.8

READ FULL TEXT
research
10/07/2021

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Deep neural networks (DNNs) are known to be vulnerable to adversarial at...
research
08/23/2023

Adversarial Training Using Feedback Loops

Deep neural networks (DNN) have found wide applicability in numerous fie...
research
06/06/2023

Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters

Adversarial attacks have been proven to be potential threats to Deep Neu...
research
08/09/2021

Neural Network Repair with Reachability Analysis

Safety is a critical concern for the next generation of autonomy that is...
research
04/03/2021

Property-driven Training: All You (N)Ever Wanted to Know About

Neural networks are known for their ability to detect general patterns i...
research
07/07/2023

A Theoretical Perspective on Subnetwork Contributions to Adversarial Robustness

The robustness of deep neural networks (DNNs) against adversarial attack...
research
03/24/2023

PIAT: Parameter Interpolation based Adversarial Training for Image Classification

Adversarial training has been demonstrated to be the most effective appr...

Please sign up or login with your details

Forgot password? Click here to reset