DeepAI AI Chat
Log In Sign Up

Better the Devil you Know: An Analysis of Evasion Attacks using Out-of-Distribution Adversarial Examples

05/05/2019
by   Vikash Sehwag, et al.
Princeton University
18

A large body of recent work has investigated the phenomenon of evasion attacks using adversarial examples for deep learning systems, where the addition of norm-bounded perturbations to the test inputs leads to incorrect output classification. Previous work has investigated this phenomenon in closed-world systems where training and test inputs follow a pre-specified distribution. However, real-world implementations of deep learning applications, such as autonomous driving and content classification are likely to operate in the open-world environment. In this paper, we demonstrate the success of open-world evasion attacks, where adversarial examples are generated from out-of-distribution inputs (OOD adversarial examples). In our study, we use 11 state-of-the-art neural network models trained on 3 image datasets of varying complexity. We first demonstrate that state-of-the-art detectors for out-of-distribution data are not robust against OOD adversarial examples. We then consider 5 known defenses for adversarial examples, including state-of-the-art robust training methods, and show that against these defenses, OOD adversarial examples can achieve up to 4× higher target success rates compared to adversarial examples generated from in-distribution data. We also take a quantitative look at how open-world evasion attacks may affect real-world systems. Finally, we present the first steps towards a robust open-world machine learning system.

READ FULL TEXT

page 2

page 12

09/06/2018

Are adversarial examples inevitable?

A wide range of defenses have been proposed to harden neural networks ag...
07/08/2020

A Critical Evaluation of Open-World Machine Learning

Open-world machine learning (ML) combines closed-world models trained on...
07/24/2017

Synthesizing Robust Adversarial Examples

Neural network-based classifiers parallel or exceed human-level accuracy...
02/18/2022

Rethinking Machine Learning Robustness via its Link with the Out-of-Distribution Problem

Despite multiple efforts made towards robust machine learning (ML) model...
01/10/2019

Characterizing and evaluating adversarial examples for Offline Handwritten Signature Verification

The phenomenon of Adversarial Examples is attracting increasing interest...
05/28/2021

Targeted Deep Learning: Framework, Methods, and Applications

Deep learning systems are typically designed to perform for a wide range...
07/24/2021

Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them

Making classifiers robust to adversarial examples is hard. Thus, many de...