adversarial-examples
Code to reproduce results in the paper "Defending against Adversarial Images using Basis Functions Transformations"
view repo
We study the effectiveness of various approaches that defend against adversarial attacks on deep networks via manipulations based on basis function representations of images. Specifically, we experiment with low-pass filtering, PCA, JPEG compression, low resolution wavelet approximation, and soft-thresholding. We evaluate these defense techniques using three types of popular attacks in black, gray and white-box settings. Our results show JPEG compression tends to outperform the other tested defenses in most of the settings considered, in addition to soft-thresholding, which performs well in specific cases, and yields a more mild decrease in accuracy on benign examples. In addition, we also mathematically derive a novel white-box attack in which the adversarial perturbation is composed only of terms corresponding a to pre-determined subset of the basis functions, of which a "low frequency attack" is a special case.
READ FULL TEXT VIEW PDF
Most deep learning models are easily vulnerable to adversarial attacks.
...
read it
Local algorithms are common tools for estimating intrinsic volumes from
...
read it
Deep neural networks are vulnerable to adversarial attacks, which can fo...
read it
Adversarial attacks to image classification systems present challenges t...
read it
This paper investigates strategies that defend against adversarial-examp...
read it
Even though deep learning have shown unmatched performance on various ta...
read it
Adversarial examples represent a great security threat for deep learning...
read it
Code to reproduce results in the paper "Defending against Adversarial Images using Basis Functions Transformations"
In the past five years, the areas of adversarial attacks (Szegedy et al., 2013)
on deep learning models, as well as defenses against such attacks, have received significant attention in the deep learning research community
(Yuan et al., 2017; Akhtar & Mian, 2018).Defenses against adversarial attacks can be categorized into two main types. Approaches of the first type modify the net training procedures or architectures, usually in order to make the net compute a smooth function; see, for example (Shaham et al., 2015; Gu & Rigazio, 2014; Cisse et al., 2017; Papernot et al., 2016b). Defenses of the second type leave the training procedure and architecture unchanged, but rather modify the data, aiming to detect or remove adversarial perturbations often by smoothing the input data. For example, Guo et al. (2017)
applied image transformations, such as total variance minimization and quilting to smooth input images.
Dziugaite et al. (2016); Das et al. (2017) proposed to apply JPEG compression to input images before feeding them through the network. Closely related approaches were taken by Akhtar et al. (2017), by applying the Discrete Cosine Transform (DCT) and by Bhagoji et al. (2017); Hendrycks & Gimpel (2017); Li & Li (2016), who proposed defense methods based on principal component analysis (PCA). De-noising using PCA, DCT and JPEG compression essentially works by representing the data using a subset of its basis functions, corresponding to the first principal components, in case of PCA, or low frequency terms, in case of DCT and JPEG. A similar idea can be applied by low-pass Fourier filtering and wavelet approximation.
In this manuscript, we continue in this direction, by investigating various defenses based on manipulations in a basis function space. Specifically, we experiment with low-pass filtering, wavelet approximation, PCA, JPEG compression, and soft-thresholding of wavelet coefficients. We apply each of these defenses as a pre-processing step on both adversarial and benign images, on the Inception-v3 and Inception-v4 networks. The defenses are applied only at test time (so that we do not re-train or change the publicly available network weights), and for each defense we evaluate its success at classifying adversarial images, as well as benign images. We evaluate these defenses in black, gray and white-box settings, using three types of popular attacks. In a black-box setting the attacker has no access to the gradients and no knowledge of the pre-processing procedure. In a gray-box setting, the attacker has access to the gradients of the attacked network, however he does not have any knowledge of defenses being applied. In a white-box setting, the attacker has access to the gradients, as well as full knowledge of the pre-processing procedure taking place. Our results show that JPEG compression performs consistently as well as and often better than the other defense approaches in defending against adversarial attacks we experimented with, across all types of adversarial attacks in black-box and gray-box settings, while also achieving high performance under two different white-box attack schemes. Soft-thresholding has the second best performance in defending against adversarial attacks, while outperforming JPEG compression on benign images.
In addition, we also mathematically derive a novel type of attack, in which the adversarial perturbation affects only terms corresponding to pre-specified subset of the basis functions. We call this approach a “filtered gradient attack”. Several cases of special interest of this attack are when this subset contains only low frequency basis functions, coarse level wavelet functions, or first principal components.
In this manuscript we experiment with three popular adversarial attacks.
Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014)
is a fast method to generate an adversarial perturbation, where the perturbation is performed by either adding or subtracting a fixed amount from each image pixel, depending on the sign of the corresponding entry in the gradient of the loss function with respect to the image. Specifically, for an image
with true label , the adversarial image is , and the adversarial perturbation is given bywhere performs elementwise and
denotes the loss of a network with parameter vector
on , viewed as a function of (i.e., holding and fixed). This perturbation can be derived from a first-order Taylor approximation of near ,maximized by choosing from a ball of radius (Shaham et al., 2015). The radius corresponds to the magnitude of the perturbation.
Iterative Fast Gradient Sign Method (I-FGSM) (Kurakin et al., 2016) works by repeated applications of the FGSM perturbation
and setting the adversarial image to be , the output of the last iteration.
Carlini-Wagner (C&W) is a family of attack methods, which typically yield perturbations of small magnitude.
They utilize a margin which enables one to generate adversarial examples which are subsequently misclassified with high confidence.
Following Guo et al. (2017), we use the C&W variant
where is a margin, which will be explained later, is a class index, is the network prediction on ,
is the logit (i.e., pre-activation of the softmax output layer) for
and is a trade-off parameter. The left-mostterm of the C&W loss corresponds to the most probable class which is not the originally predicted one. For
, the left part of the loss is minimized when there is a class which is not the original predicted class , and whose logit is at least as big as the logit of the true class. Increasing the margin requires that the gap between the logits increases correspondingly, resulting in a high confidence adversarial example. In our experiments we use . The fact that C&W perturbations are typically small in magnitude is a result of minimizing the squared difference . Unlike FGSM and I-FGSM examples, C&W examples are much slower to generate, requiring applying an optimizer for each adversarial example.Figure 1 shows a benign image, adversarial images generated using the FGSM, I-FGSM and C&W methods and the corresponding perturbations generated by those methods. The adversarial perturbations were generated using the Inception-v3 network.
We experiment with several defense methods, all of which operate by performing manipulations in basis function spaces. Below we describe each of the defenses used in our experiments. Further technical details are given in Section 4.1.
Low-pass filtering:
The discrete Fourier transform of a two-dimensional signal (e.g., an image)
is defined bywhere , , and .
Low pass-filtering is performed by obtaining the Fourier representation of the image, followed by element-wise multiplication of the Fourier coefficients with a bump function, so that high frequencies are depressed, and lastly converting the signal back to pixel space using the inverse transform.
PCA:
PCA de-noising represents a given matrix by a low-rank approximation of it, while keeping as much variance as possible in the original data matrix (whose rows are viewed as data points and columns as input features).
This is done by obtaining the principal components of a matrix, representing the data in the PC space, discarding all but the leading principal directions and mapping the data back to its input space. Mathematically, this procedure is formalized by
where is matrix where each row corresponds to a data point, and is a matrix containing the leading eigenvectors of the covariance matrix . Rather than computing the PCA on the entire image dataset, as was done by Bhagoji et al. (2017); Hendrycks & Gimpel (2017); Li & Li (2016), we compute the principal components for each image separately, in two different ways:
Viewing the image as a matrix of size , i.e., where rows are considered as data points, and performing PCA denoising on that matrix.
We cut patches from each image, re-shape each patch to a vector and obtain a matrix whose rows are the patch vectors. We then perform PCA denoising on that matrix.
in both cases, we apply the denoising on each color channel separately.
Wavelet approximation:
Unlike complex exponentials, the Fourier basis functions, wavelet basis functions are localized.
Wavelet basis on is an orthonormal collection of zero-mean functions, created from a bump function (“father wavelet”) via
and
The index corresponds to the level of approximation (via the width of the bump) and to the shift. Wavelet decomposition of a real-valued signal is represented as sequences of coefficients, where the th sequence describes the change of the signal at the th level, and . is then represented as
Discrete wavelet transform is a wavelet transform where the wavelets are discretely sampled. Since wavelet functions are localized, a wavelet basis is often better than the Fourier basis at representing non-smooth signals, yielding sparser representations. For 2D images, level wavelet approximation results in an approximation image of resolution which is coarser as grows, containing
of the pixels of the original image. To resize the approximation image back to the original size, we use bi-cubic interpolation, implemented via Matlab’s
imresize function.s are iid standard Gaussian random variables,
and . They propose to de-noise using soft-thresholding of its wavelet coefficients, where the soft-thresholding operator is defined byand t is a threshold, usually chosen to be
A classical result by Donoho (1995) proves that such de-noising is min-max optimal in terms of distance between the de-noised signal and the original one, while keeping at least as smooth as .
JPEG compression:
JPEG is a lossy compression that utilizes DCT and typically removes many high frequency components, to which human perception is less sensitive.
Specifically, JPEG compression consists of the following steps:
Conversion of the image from RGB format to format, where the channel represents Luminance and channels represent chrominance.
Down-sampling of the chrominance channels.
Splitting of the image to blocks and applying 2D DCT to each block. This is done for each channel separately.
Quantization of the frequency amplitudes, achieved by dividing each frequency term by a (different) constant and rounding it to the nearest integer. As a result, many high frequency components are typically set to zero, and others are shrinked. The amount of compression is governed by a user-specified quality parameter, defining the reduction in resolution.
Lossless compression of the block data.
The lossy elements of JPEG compression are the down-sampling (step 2) and the quantization (step 3), where most of the compression is achieved.
JPEG defense was applied by Dziugaite et al. (2016); Das et al. (2017); Guo et al. (2017).
In the case of color images, Fourier and wavelet transforms are typically applied on each color channel separately.
Figure 2 demonstrates the above defense methods on the panda image of Figure 1.
Our experiments were performed on the publicly available dataset from the NIPS 2017 security competition^{2}^{2}2https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack/data, containing 1000 images, as well as a trained Inception-v3 model. All attacks were carried out using Cleverhans^{3}^{3}3https://github.com/tensorflow/cleverhans/. For C&W we used with and , and similarly to (Guo et al., 2017), the perturbations were multiplied by constant to alter their magnitude. FGSM and I-FGSM attacks were performed with . The parameters of each of the defenses were selected to optimize the performance of the defense in a gray-box setting. Specifically, the low-pass filtering was applied by multiplying the Fourier coefficients of each color channel with a circle with radius of 65; PCA was performed by retaining the largest 36 principal components of each image; Patchwise PCA was performed on patches of size , and retaining the largest 13 principal components; JPEG compression was performed by setting the quality parameter to 23%. Wavelet approximation was performed in Matlab using the appcoef2 command; soft-thresholding was done using Matlab’s ddencmp and wdencmp; for both approaches we used the bi-orthogonal bior3.1 wavelet. In all experiments the adversarial examples were generated using the Inception-v3 network. We did not perform any re-training or fine-tuning of the net. All defenses were applied as test time pre-processing methods. Our codes are available at https://github.com/KlugerLab/adversarial-examples.
Following Guo et al. (2017), for each defense we report the top-1 accuracy versus the normalized norm of the perturbation, defined by
where and denote benign and adversarial examples respectively, and is the number of examples.
In this setting, the attacker has no access to the gradients of the target network. Rather, the attack is based on the transferability property of adversarial examples (Liu et al., 2016; Papernot et al., 2016a). Specifically, we generated adversarial examples generated for Inception-v3, applied each of the defenses (separately) on each adversarial example, fed them into Inception-v4, and measured the top 1 accuracy. Figure 3 shows the performance of each defense method as a function of the normalized norm of the perturbation, for each of the attack methods. Overall, we found that in our experiment setup, transferability actually requires fairly large perturbations comparing to a gray-box setting (see Section 4.4)^{4}^{4}4We found that the largest normalized we consider in the black-box case corresponds to about of the one used by Liu et al. (2016). We chose not to use larger perturbations as these become fairly noticeable to a human eye, and hence less adversarial.. Adversarial examples with perturbations of normalized norm below 0.08 generally only yield a modest decrease in accuracy. Consequently, all the tested defenses are ineffective against small perturbations in this setting. JPEG denoising becomes effective around 0.08 against FGM and I-FGM attacks, where it outperforms all other defenses, however does not perform well against C&W attacks. Soft-thresholding becomes effective against I-FGM and C&W attacks for norms above 0.08. Low-pass filtering and the PCA denoising methods do not perform well against any attack. In addition, Table 1 displays the performance of each of the defenses on benign examples using Inception-v4.
Defense | Top 1 accuracy |
---|---|
No defense | 0.967 |
Low-pass filter | 0.89 |
PCA | 0.853 |
JPEG | 0.912 |
Level 1 wavelet approximation | 0.933 |
Soft-thresholding | 0.941 |
Patchwise PCA | 0.811 |
As can be seen, the wavelet-based methods (level 1 approximation and soft-thresholding yield the smallest decrease in accuracy on benign examples, followed by JPEG. Low-pass filtering and PCA methods yield a more significant decrease in accuracy.
In this setting, the attacker has access to the gradients of the target network, but is not aware of the defenses applied. Specifically, we used the FGSM, I-FGSM and C&W examples generated for the Inception-v3 network, applied each of the defenses (separately) on each adversarial example, fed them back into Inception-v3, and measured the top 1 accuracy. The results are shown in Figure 4.
As can be seen, JPEG denoising performs for the most part as well as or better than all other methods, consistently across all attacks and all perturbation magnitudes. In the FGSM case, as the magnitude of the perturbation gets large, soft-thresholding and patchwise PCA outperform JPEG denoising.
To complete the evaluation of the defenses in this setting, we also measure the performance of each of the defenses on benign examples, which is shown in Table 2.
Defense | Top 1 accuracy |
---|---|
No defense | 0.956 |
Low-pass filter | 0.855 |
PCA | 0.796 |
JPEG | 0.883 |
Level 1 wavelet approximation | 0.901 |
Soft-thresholding | 0.911 |
Patchwise PCA | 0.73 |
In this setting, the attacker has access to the gradients of the target network, and also has full knowledge of the defense being applied.
Below we present two specific schemes where we utilize this knowledge.
Filtered Gradient Attack (FGA):
Let , and let be an orthonormal set of basis functions on (e.g., principal components, complex exponentials or wavelet functions).
Write , where and are the subsets of retained and filtered basis functions, respectively, and .
We can write using the basis functions as
where are vectors describing the coefficients of each basis function in the representation of . Let be the loss of a neural net with parameter for the example , and let be its corresponding gradient w.r.t . A Filtered Gradient Attack would only modify . This can be done by computing the gradient of the loss w.r.t
, using the chain rule:
(1) |
The gradient in (1) is defined in the -dimensional space of functions in . To map it back to , one should multiply it from the left by , which defines the adversarial perturbation in the input space as
(2) |
Equation (2) simply describes a filtered gradient , hence the attack name. Some cases of special interest are where the retained basis functions correspond to low frequency terms, first principal components, or coarse wavelet functions; in these cases the FGA perturbation is smoother than usual adversarial perturbations. More generally, we can apply any of the de-noising procedures in this manuscript on the gradient, to obtain a smooth adversarial perturbation. In this section we apply each of the de-noising procedures in this manner within a FGSM attack, which results in the following procedure, applied to a raw image :
Forward-propagate through the net, and compute its loss.
Obtain the gradient of the loss w.r.t .
De-noise the gradient to get .
.
Backward Pass Differentiable Approximation (BPDA): This attack was proposed in Athalye et al. (2018) for cases where it is hard or impossible to compute the gradient of a pre-processor which is applied as defense. Specifically, we can view the de-noising defense as the first layer of the neural net, which performs pre-processing of the input. When this pre-processing is differentiable, standard attacks can be utilized. When it is impossible to compute the gradient of the pre-processing, Athalye et al. propose to approximate it using the identity function, which they justify since the pre-processing step computes a function . We apply this logic within a FGSM framework, which results in the following procedure, applied to a raw image :
De-noise using any of the defense methods to get
Forward-propagate the through the net, and compute its loss.
Obtain the gradient of the loss w.r.t to the de-noised image.
.
We tested the FGA and BPDA using all defense techniques considered in this work; the results are shown in Figure 5.
As can be seen, JPEG appears to be the most successful defense among all tested defenses, under both attack schemes.
We explored various pre-processing techniques as defenses against adversarial attacks by applying them as test-time pre-processing procedures and measuring their performance under gray, black and white-box settings. Our results empirically show that in a black-box setting, JPEG compression and soft-thresholding perform best, while the former outperforms all other tested defenses in gray-box and the two white-box setting considered. In addition, we proposed the Filtered Gradient Attack, a novel white-box attack scheme, where only components corresponding to a pre-defined basis functions are changed. A special case of FGA is a ’low-frequency’ attack.
International Conference on Machine Learning
, pp. 854–863, 2017.Distillation as a defense to adversarial perturbations against deep neural networks.
In Security and Privacy (SP), 2016 IEEE Symposium on, pp. 582–597. IEEE, 2016b.
Comments
There are no comments yet.