Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples

05/05/2019
by   Vikash Sehwag, et al.
18

A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follow a pre-specified distribution. However, real-world implementations of deep learning applications, such as autonomous driving and content classification are likely to operate in the open-world environment. In this paper, we demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out-of-distribution inputs (OOD adversarial examples). In our study, we use 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity. We first demonstrate that state-of-the-art detectors for out-of-distribution data are not robust against OOD adversarial examples. We then consider 5 known defenses for adversarial examples, including state-of-the-art robust training methods, and show that against these defenses, OOD adversarial examples can achieve up to 4× higher target success rates compared to adversarial examples generated from in-distribution data. We also take a quantitative look at how open-world evasion attacks may affect real-world systems. Finally, we present the first steps towards a robust open-world machine learning system.

READ FULL TEXT

page 2

page 12

research
09/06/2018

Are adversarial examples inevitable?

A wide range of defenses have been proposed to harden neural networks ag...
research
07/08/2020

A Critical Evaluation of Open-World Machine Learning

Open-world machine learning (ML) combines closed-world models trained on...
research
07/24/2017

Synthesizing Robust Adversarial Examples

Neural network-based classifiers parallel or exceed human-level accuracy...
research
02/18/2022

Rethinking Machine Learning Robustness via its Link with the Out-of-Distribution Problem

Despite multiple efforts made towards robust machine learning (ML) model...
research
01/10/2019

Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification

The phenomenon of Adversarial Examples is attracting increasing interest...
research
05/28/2021

Targeted Deep Learning: Framework, Methods, and Applications

Deep learning systems are typically designed to perform for a wide range...
research
07/24/2021

Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them

Making classifiers robust to adversarial examples is hard. Thus, many de...

Please sign up or login with your details

Forgot password? Click here to reset