DeepAI AI Chat
Log In Sign Up

Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense

02/02/2023
by   Zunzhi You, et al.
The University of Sydney
0

Masked Image Modeling (MIM) has been a prevailing framework for self-supervised visual representation learning. Within the pretraining-finetuning paradigm, the MIM framework trains an encoder by reconstructing masked image patches with the help of a decoder which would be abandoned when the encoder is used for finetuning. Despite its state-of-the-art performance on clean images, MIM models are vulnerable to adversarial attacks, limiting its real-world application, and few studies have focused on this issue. In this paper, we have discovered that noisy image modeling (NIM), a variant of MIM that uses denoising as the pre-text task, provides not only good pretrained visual features, but also effective adversarial defense for downstream models. To achieve a better accuracy-robustness trade-off, we further propose to sample the hyperparameter that controls the reconstruction difficulty from random distributions instead of setting it globally, and fine-tune downstream networks with denoised images. Experimental results demonstrate that our pre-trained denoising autoencoders are effective against different white-box, gray-box, and black-box attacks without being trained with adversarial images, while not harming the clean accuracy of fine-tuned models. Source code and models will be made available.

READ FULL TEXT

page 1

page 3

page 6

page 8

02/19/2018

Divide, Denoise, and Defend against Adversarial Attacks

Deep neural networks, although shown to be a successful class of machine...
08/28/2022

Cross-domain Cross-architecture Black-box Attacks on Fine-tuned Models with Transferred Evolutionary Strategies

Fine-tuning can be vulnerable to adversarial attacks. Existing works abo...
03/19/2022

Adversarial Defense via Image Denoising with Chaotic Encryption

In the literature on adversarial examples, white box and black box attac...
06/05/2020

Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning

High-performance anti-spoofing models for automatic speaker verification...
03/21/2023

Black-box Backdoor Defense via Zero-shot Image Purification

Backdoor attacks inject poisoned data into the training set, resulting i...
02/12/2022

Open-set Adversarial Defense with Clean-Adversarial Mutual Learning

Open-set recognition and adversarial defense study two key aspects of de...
08/04/2022

Privacy Safe Representation Learning via Frequency Filtering Encoder

Deep learning models are increasingly deployed in real-world application...