Towards Transferable Adversarial Attacks on Vision Transformers

09/09/2021
by   Zhipeng Wei, et al.
0

Vision transformers (ViTs) have demonstrated impressive performance on a series of computer vision tasks, yet they still suffer from adversarial examples. In this paper, we posit that adversarial attacks on transformers should be specially tailored for their architecture, jointly considering both patches and self-attention, in order to achieve high transferability. More specifically, we introduce a dual attack framework, which contains a Pay No Attention (PNA) attack and a PatchOut attack, to improve the transferability of adversarial samples across different ViTs. We show that skipping the gradients of attention during backpropagation can generate adversarial examples with high transferability. In addition, adversarial perturbations generated by optimizing randomly sampled subsets of patches at each iteration achieve higher attack success rates than attacks using all patches. We evaluate the transferability of attacks on state-of-the-art ViTs, CNNs and robustly trained CNNs. The results of these experiments demonstrate that the proposed dual attack can greatly boost transferability between ViTs and from ViTs to CNNs. In addition, the proposed method can easily be combined with existing transfer methods to boost performance.

READ FULL TEXT
research
04/27/2022

Improving the Transferability of Adversarial Examples with Restructure Embedded Patches

Vision transformers (ViTs) have demonstrated impressive performance in v...
research
03/10/2023

Boosting Adversarial Attacks by Leveraging Decision Boundary Information

Due to the gap between a substitute model and a victim model, the gradie...
research
03/28/2023

Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization

Vision transformers (ViTs) have been successfully deployed in a variety ...
research
06/08/2021

On Improving Adversarial Transferability of Vision Transformers

Vision transformers (ViTs) process input images as sequences of patches ...
research
11/30/2018

Transferable Adversarial Attacks for Image and Video Object Detection

Adversarial examples have been demonstrated to threaten many computer vi...
research
06/28/2023

Boosting Adversarial Transferability with Learnable Patch-wise Masks

Adversarial examples have raised widespread attention in security-critic...
research
12/21/2020

On Success and Simplicity: A Second Look at Transferable Targeted Attacks

There is broad consensus among researchers studying adversarial examples...

Please sign up or login with your details

Forgot password? Click here to reset