Universal Adversarial Perturbations Against Semantic Image Segmentation

04/19/2017 ∙ by Jan-Hendrik Metzen, et al. ∙ Bosch University of Freiburg 0

While deep learning is remarkably successful on perceptual tasks, it was also shown to be vulnerable to adversarial perturbations of the input. These perturbations denote noise added to the input that was generated specifically to fool the system while being quasi-imperceptible for humans. More severely, there even exist universal perturbations that are input-agnostic but fool the network on the majority of inputs. While recent work has focused on image classification, this work proposes attacks against semantic image segmentation: we present an approach for generating (universal) adversarial perturbations that make the network yield a desired target segmentation as output. We show empirically that there exist barely perceptible universal noise patterns which result in nearly the same predicted segmentation for arbitrary inputs. Furthermore, we also show the existence of universal noise which removes a target class (e.g., all pedestrians) from the segmentation while leaving the segmentation mostly unchanged otherwise.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 11

page 12

page 13

page 14

page 15

page 16

page 17

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

While deep learning has led to significant performance increases for numerous visual perceptual tasks [10, 14, 20, 25] and is relatively robust to random noise [6], several studies have found it to be vulnerable to adversarial perturbations [24, 9, 17, 22, 2]

. Adversarial attacks involve generating slightly perturbed versions of the input data that fool the classifier (i.e., change its output) but stay almost imperceptible to the human eye. Adversarial perturbations transfer between different network architectures, and networks trained on disjoint subsets of data

[24]. Furthermore, Papernot et al. [18] showed that adversarial examples for a network of unknown architecture can be constructed by training an auxiliary network on similar data and exploiting the transferability of adversarial examples.

(a) Image (b) Prediction
(c) Adversarial Example (d) Prediction
Figure 1: The upper row shows an image from the validation set of Cityscapes and its prediction. The lower row shows the image perturbed with universal adversarial noise and the resulting prediction. Note that the prediction would look very similar for other images when perturbed with the same noise (see Figure 3).

Prior work on adversarial examples focuses on the task of image classification. In this paper, we investigate the effect of adversarial attacks on tasks involving a localization component, more specifically: semantic image segmentation. Semantic image segmentation is an important methodology for scene understanding that can be used for example for automated driving, video surveillance, or robotics. With the wide-spread applicability in those domains comes the risk of being confronted with an adversary trying to fool the system. Thus, studying adversarial attacks on semantic segmentation systems deployed in the physical world becomes an important problem.

Adversarial attacks that aim at systems grounded in the physical world should be physically realizable and inconspicuous [22]. One prerequisite for physical realizability is that perturbations do not depend on the specific input since this input is not known in advance when the perturbations (which need to be placed in the physical world) are determined. This work proposes a method for generating image-agnostic universal perturbations. Universal perturbations have been proposed by Moosavi-Dezfooli et al. [16]; however, we extend the idea to the task of semantic image segmentation. We leave further prerequisites for physical realizability as detailed by Sharif et al. [22] to future work.

An attack is inconspicuous if it does not raise the suspicion of humans monitoring the system (at least not under cursory investigation). This requires that the system inputs are modified only subtly, and, for a semantic image segmentation task, also requires that system output (the scene segmentation) looks mostly as a human would expect for the given scene. If an adversary’s objective is to remove all occurrences of a specific class (e.g., an adversary trying to hide all pedestrians to deceive an emergency braking system) then the attack is maximally inconspicuous if it leaves the prediction for all other classes unchanged and only hides the target class. We present one adversarial attack which explicitly targets this dynamic target segmentation scenario.

While inconspicuous attacks require that target scenes mostly match what a human expects, we also present an attack yielding an static target segmentations. This attack generates universal perturbations that let the system output always essentially the same segmentation regardless of the input, even when the input is from a completely different scene (see Figure 1). The main motivation for this experiment is to show how fragile current approaches for semantic segmentation are when confronted with an adversary. In practice, such attacks could be used in scenarios in which a static camera monitors a scene (for instance in surveillance scenarios) as it would allow an attacker to always output the segmentation of the background scene and blend out all activity like, e.g., burglars robbing a jewelry shop.

We summarize our main contributions as follows:

  • We show the existence of (targeted) universal perturbations for semantic image segmentation models. Their existence was not clear a priori because the receptive fields of different output elements largely overlap. Thus perturbations cannot be chosen independently for each output target. This makes the space of adversarial perturbations for semantic image segmentation presumably smaller than for recognition tasks like image classification and the existence of universal perturbations even more surprising.

  • We propose two efficient methods for generating these universal perturbations. These methods optimize the perturbations on a training set. The objective of the first methods is to let the network yield a fixed target segmentation as output. The second method’s objective is to leave the segmentation unchanged except for removing a designated target class.

  • We show empirically that the generated perturbations are generalizable: they fool unseen validation images with high probability. Controlling the capacity of universal perturbations is important for achieving this generalization from small training sets.

  • We show that universal perturbations generated for a fixed target segmentation have a local structure that resembles the target scene (see Figure 4).

2 Background

Let be a function with parameters . Moreover, let be an input of , be the output of , and be the corresponding ground-truth target. More specifically for the scenario studied in this work,

denotes a deep neural network,

an image, the conditional probability

encoded as a class probability vector, and

a one-hot encoding of the class. Furthermore, let

be the basic classification loss such as cross-entropy. We assume that is differentiable with respect to and with respect to .

2.1 Semantic Image Segmentation

Semantic image segmentation denotes a dense prediction task that addresses the “what is where in an image?” question by assigning a class label to each pixel of the image. Recently, deep learning based approaches (oftentimes combined with conditional random fields) have become the dominant and best performing class of methods for this task [14, 13, 30, 3, 28, 4]. In this work, we focus on one of the first and most prominent architectures, the fully convolutional network architecture FCN-8s introduced by Long et al. [14] for the VGG16 model [23].

The FCN-8s architecture can roughly be divided into two parts: an encoder part which transforms a given image into a low resolution semantic representation and a decoder part which increases the localization accuracy and yields the final semantic segmentation at the resolution of the input image. The encoder part is based on a VGG16 pretrained on ImageNet

[21]

where the fully connected layers are reinterpreted as convolutions making the network “fully convolutional”. The output of the last encoder layer can be interpreted as a low-resolution semantic representation of the image and is the input to five upsampling layers which recover the high spatial resolution of the image via successive bilinear-interpolation (FCN-32s). For FCN-8s, additionally two parallel paths merge higher-resolution, less abstract layers of the VGG16 into the upsampling path via convolutions and element-wise summation. This enables the network to utilize features with a higher spatial resolution.

2.2 Adversarial Examples

Let denote an adversarial perturbation for an input and let denote the corresponding adversarial example. The objective of an adversary is to find a perturbation which changes the output of the model in a desired way. For instance the perturbation can either make the true class less likely or a designated target class more likely. At the same time, the adversary typically tries to keep quasi-imperceptible by, e.g., bounding its -norm.

The first method for generating adversarial examples was proposed by Szegedy et al. [24]. While this method was able to generate adversarial examples successfully for many inputs and networks, it was also relatively slow computationally since it involved an L-BFGS-based optimization. Since then, several methods for generating adversarial examples have been proposed. These methods either maximize the predicted probability for all but the true class or minimize the probability of the true class.

Goodfellow et al. [9] proposed a non-iterative and hence fast method for computing adversarial perturbations. This fast gradient-sign method (FGSM) defines an adversarial perturbation as the direction in image space which yields the highest increase of the linearized cost function under -norm. This can be achieved by performing one step in the gradient sign’s direction with step-width :

Here, is a hyper-parameter governing the distance between original image and adversarial image. FGSM is a targeted method. This means that the adversary is solely trying to make the predicted probability of the true class smaller. However, it does not control which of the other classes becomes more probable.

Kurakin et al. [11] proposed an extension of FGSM which is iterative and targeted. The proposed least-likely method (LLM) makes the least likely class under the prediction of the model more probable. LLM is in principle not specific for the least-likely class ; it can rather be used with an arbitrary target class . The method tries to find which maximizes the predictive probability of class under . This can be achieved by the following iterative procedure:

Here denotes a step size and all entries of are clipped after each iteration such that their absolute value remains smaller than . We use throughout all experiments. Concurrent with this work, adversarial examples have been extended to semantic image segmentation and object detection [27, 8]. Moreover, training with adversarial examples has been applied to mammographic mass segmentation to reduce overfitting [32].

For the methods outlined above, the adversarial perturbation depends on the input . Recently, Moosavi-Dezfooli et al. [16] proposed a method for generating universal, image-agnostic perturbations that, when added to arbitrary data points, fool deep nets on a large fraction of images. The method for generating these adversarial perturbations is based on the adversarial attack method DeepFool [17]. DeepFool is applied to a set of images (the train set). These images are presented sequentially in a round-robin manner to DeepFool. For the first image, DeepFool identifies a standard image-dependent perturbation. For subsequent images, it is checked whether adding the previous adversarial perturbation already fools the classifier; if yes the algorithm continues with the next image, otherwise it updates the perturbation using DeepFool such that also the current image becomes adversarial. The algorithm stops once the perturbation is adversarial on a large fraction of the train set.

The authors show impressive results on ImageNet [21], where they show that the perturbations are adversarial for a large fraction of test images, which the method did not see while generating the perturbation. One potential short-coming of the approach is that the attack is not targeted, i.e., the adversary cannot control which class the classifier shall assign to an adversarial example. Moreover, for high-resolution images and a small train set, the perturbation might overfit the train set and not generalize to unseen test data since the number of “tunable parameters” is proportional to the number of pixels. Thus, high-resolution images will need a large train set and a large computational budget. In this paper, we propose a method which overcomes these shortcomings.

3 Adversarial Perturbations Against Semantic Image Segmentation

For semantic image segmentation, the loss is a sum over the spatial dimensions of the target such as:

In this section, we describe how to find an input for such that becomes minimal, i.e., how an adversary can do quasi-imperceptible changes to the input such that it achieves a desired target segmentation . We start by describing how an adversary can choose .

3.1 Adversarial Target Generation

In principle, an adversary may choose arbitrarily. Crucially, however, an adversary may not choose based on since the ground-truth is also unknown to the adversary. Instead, the adversary may use as basis as we assume that the adversary has access to .

As motivated in Section 1, typical scenarios involve an adversary whose primary objective is to hide certain kinds of objects such as, e.g., pedestrians. As a secondary objective, an adversary may try to perform attacks that are inconspicuous, i.e., do not call the attention of humans monitoring the system (at least not under cursory investigation) [22]. Thus the input must be modified only subtly. For a semantic image segmentation task, however, it is also required that the output of the system looks mostly as a human would expect for the given scene. This can be achieved, for instance, by keeping as similar as possible to where the primary objective does not apply. We define two different ways of generating the target segmentation:

Figure 2: Illustration of an adversary generating a dynamic target segmentation for hiding pedestrians.

Static target segmentation:

In this scenario, the adversary defines a fixed segmentation, such as the system’s prediction at a time step , as target for all subsequent time steps: . This target segmentation is suited for instance in situations where an adversary wants to attack a system based on a static camera and wants to hide suspicious activity in a certain time span that had not yet started at time .

Dynamic target segmentation:

In situations involving ego-motion, a static target segmentation is not suited as it would not account for changes in the scene caused by the movement of the camera. In contrast, dynamic target segmentation aims at keeping the network’s segmentation unchanged with the exception of removing certain target classes. Let be the class of objects the adversary wants to hide, and let and . We assign for all , and for all with . The latter corresponds to filling the gaps left in the target segmentation by removing elements predicted to be

using a nearest-neighbor heuristic. An illustration of the adversarial target generation is shown in Figure

2.

3.2 Image-Dependent Perturbations

Before turning to image-agnostic universal perturbations, we first define how an adversary might choose an image-dependent perturbation. Given , we formulate the objective of the adversary as follows:

The constraint limits the adversarial example to have at most an -distance of to . Let implement the constraint by clipping all entries of to have at most an absolute value of . Based on this, we can define a targeted iterative adversary analogously to the least-likely method (see Section 2.2):

An alternative formulation which takes into consideration that the primary objective (hiding objects) and the secondary objective (being inconspicuous) are not necessarily equally important can be achieved by a modified version of the loss including a weighting parameter :

Here, lets the adversary solely focus on removing target-class predictions, forces the adversary only to keep the background constant, and for .

An additional issue for (and ) is that there is potentially competition between different target pixels, i.e., the gradient of the loss for might point in the opposite direction as the loss gradient for

. Standard classification losses such as the cross entropy in general encourage target predictions which are already correct to become more confident as this reduces the loss. This is not necessarily desirable in face of competition between different targets. The reason for this is that loss gradients for making correct predictions more confident might counteract loss gradients which would make wrong predictions correct. Note that this issue does not exist for adversaries targeted at image classification as there is essentially only a single target output. To address this issue, we set the loss of target pixels which are predicted as the desired target with a confidence above

to [26]. Throughout this paper, we use .

3.3 Universal Perturbations

In this section, we propose a method for generating universal adversarial perturbations in the context of semantic segmentation. The general setting is that we generate on a set of training inputs , where was generated with either of the two methods presented in Section 3.1. We are interested in the generalization of to test inputs for which it was not optimized and for which no target exists. This generalization to inputs for which no target exists is required because generating would require evaluating which might not be possible at test time or under real-time constraints. We propose the following extension of the attack presented in Section 3.2:

with being the loss gradient averaged over the entire training data. A potential issue of this approach is overfitting to the training data which would reduce generalization of to unseen inputs. Overfitting is actually likely given that has the same dimensionality as the input image and is thus high-dimensional. We adopt a relatively simple regularization approach by enforcing to be periodic in both spatial dimensions. More specifically, we enforce for all the constraints and for a predefined spatial periodicity . This can be achieved by optimizing a proto-perturbation of size and tile it to the full . This results in a gradient averaged over the training data and all tiles:

with , denoting the number of tiles per dimension and .

As we will show in Section 4, the quality of the generated universal perturbation depends crucially on the size of the train set. As our method for generating universal perturbations does not require ground-truth labels, we may in principle use arbitrary large unlabeled data sets. Nevertheless, we also investigate how well universal perturbations can be generated for small since large requires considerable computational resources and also more queries to , which might increase monetary costs or the risk of being identified.

4 Experimental Results

We evaluated the proposed adversarial attacks against semantic image segmentation on the Cityscapes dataset [5], which consists of publicly available labeled RGB images ( for training and for validation) with a resolution of pixels from different cities. We used the pixel-wise fine annotations covering frequent classes. For computational reasons, all images and labels were downsampled to a resolution of pixels, where for images a bilinear interpolation and for labels a nearest-neighbor approach was used for down-sampling. We trained the FCN-8s network architecture (see Section 2.1) for semantic image segmentation on the whole training data and achieved a class-wise intersection-over-union (IoU) on the validation data of 64.8%.

We generated the universal perturbations on (subsets of) the training data and evaluated them on unseen validation data. When not noted otherwise, we used in the experiments. This value of was also used by Moosavi-Dezfooli et al. [16] and corresponds to a level of noise which is only perceptible for humans at closer inspection. Moreover, we set the number of iterations to .

Static Target Segmentation

As Cityscapes does not involve static scenes, we evaluated an even more challenging scenario: namely to output a static target scene segmentation which has nothing in common with the actual input scene present in the image. For this, we selected an arbitrary ground-truth segmentation (monchengladbach_000000_026602_gtFine) from Cityscapes as target. We set the number of training images to , which corresponds to the number of images in the Cityscapes train set. Moreover, we used the unweighted loss , and did not use periodic tiles, i.e., . An illustration for this setting on unseen validation images is shown in Figure 3. The adversary achieved the desired target segmentation nearly perfectly when adding the universal perturbation that was generated on the training images. This is even more striking as for a human, the original scene, which has nothing in common with the target scene, remains clearly dominant.

(a) image 1 (b) pred. image 1 (c) image 2 (d) pred. image 2

(e) universal noise (4x)
(f) static adv. target (g) universal noise (4x) (h) static adv. target

(i) adv. example 1
(j) pred. adv. 1 (k) adv. example 2 (l) pred. adv. 2
Figure 3: Influence of universal adversarial perturbation for static targets (: (a) First unmodified Cityscapes image. (b) Network prediction on (a) (c) Second unmodified Cityscapes image. (d) Network prediction on (c) (e) Universal adversarial perturbation (amplified by factor 4). (f) Static adversarial target. (g) Universal adversarial perturbation (same as (e)). (h) Static adversarial target (same as (f)). (i) Adversarial example for (a). (j) Network prediction on (i) (k) Adversarial example for (c). (l) Network prediction on (k). Please refer to the supplementary material (Section B-D) for additional and higher-resolution illustrations.

Figure 4 shows an illustration of the generated universal perturbation for . This perturbation is highly structured and the local structure depends strongly on the target class. When comparing the perturbation with the static target segmentation, it is fairly easy to recognize the structure of the target in the perturbation. For instance, man-made structures such as buildings and fences correspond to mostly horizontal and vertical edges. This property indicates that the adversarial attack might exploit the (generally desirable) robustness of deep networks to contrast changes. This allows low contrast noise structures to have stronger impact than the high-contrast structures in the actual image.

Figure 4: Illustration of universal perturbation for a static target segmentation (, not amplified). Best seen in color. The network’s prediction when applied to the perturbation itself as input strongly resembles the static target segmentation (see Section H).

Table 1 shows a quantitative analysis of the success rate for different values of . Here, we define the success rate as the categorical accuracy between static target segmentation and predicted segmentation of the network on the adversarial example. The success rate on training and validation data is nearly on par, which shows that overfitting is not an issue even for high-dimensional perturbations. This is probably due to the large number of training images and the consistent target. Unsurprisingly, larger leads to higher success rates. The value strikes a good balance between high success rate and being quasi-imperceptible.

2 5 10 20
Training data 60.9% 82.0% 92.7% 97.2%
Validation data 60.9% 80.3% 91.0% 96.3%
Table 1: Success rate of static target segmentation for different values of . The generated perturbations achieve nearly the same success rate on unseen validation data as on the training data.

Dynamic Target Segmentation

In this experiment, we focused on an adversary which tries to hide all pedestrians (Cityscapes class “person”) in an image while leaving the segmentation unchanged otherwise. When not noted otherwise, we set the number of training images to (this value corresponds to the number of images containing pedestrians in the Cityscapes train set), the periodic tile size to and use with as motivated empirically (see Figure 6 and Table 2 and 3). An illustration for this setting on unseen validation images is shown in Figure 5. We note that qualitatively, the adversary succeeds in removing nearly all pedestrian pixels while leaving the background mostly unchanged. However, closer inspection by a human would probably raise suspicion as the predicted segmentation looks relatively inhomogeneous.

(a) image 1 (b) pred. image 1 (c) image 2 (d) pred. image 2

(e) universal noise (4x)
(f) dynamic adv. target 1 (g) universal noise (4x) (h) dynamic adv. target 2

(i) adv. example 1
(j) pred. adv. 1 (k) adv. example 2 (l) pred. adv. 2
Figure 5: Influence of universal adversarial perturbation for dynamic targets (): (a) First unmodified Cityscapes image. (b) Network prediction on (a). (c) Second unmodified Cityscapes image. (d) Network prediction on (c). (e) Universal adversarial perturbation (amplified by factor 4). (f) Dynamic adversarial target for (a). Note that the adversary does not tailor the universal perturbation to this target for validation data; the image solely shows the ideal output. (g) Universal adversarial perturbation (same as (e)). (h) Dynamic adversarial target for (c). (i) Adversarial example for (a). (j) Network prediction on (i). (k) Adversarial example for (c). (l) Network prediction on (k). Please refer to the supplementary material (Section E-G) for additional and higher-resolution illustrations.

For quantifying how well an adversary achieves its primary objective of hiding a target class, we measure which percentage of the pixels that were predicted as pedestrians on the original input are assigned to any of the other classes for the adversarial example (“Pedestrian pixels hidden”). We measure the categorical accuracy on background pixels (i.e., pixels that were not predicted as pedestrians on the original input) between dynamic adversarial target segmentation and the segmentation predicted by the network on the adversarial example (“Background pixels preserved”). This quantifies the secondary objective of being inconspicuous by preserving the background. Note that this comparison does not involve the ground-truth segmentation; we solely measure if the network’s original background segmentation is preserved.

Figure 6: Evaluation of universal perturbations on dynamic target segmentation for different tile-sizes and number of train images (between 100 and 1700) on validation data (, ). More training images improve generalization to validation data. Smaller tile sizes increase the percentage of pedestrian pixels removed at the cost of preserving the background less well. For comparison, image-dependent non-periodic perturbations are also shown, which nearly perfectly achieve both objectives.
2 5 10 20
Pedestrian hidden 40% 93% 100% 100%
Background pres. 95% 84% 87% 89%
Pedestrian hidden 34% 81% 92% 93%
Background pres. 94% 85% 86% 87%
Table 2: Dynamic target for different values of on training data (top) and validation data (bottom).
no 0.9 0.99 0.999 0.9999
Pedestrian hidden 41% 70% 83% 88% 92%
Background pres. 96% 94% 91% 89% 86%
Table 3: Dynamic target for different values of on validation data.

Figure 6 shows how the periodic tile-size and , the number of training images, affects the results of the adversary. In general, more training images and smaller tile-sizes increase the number of hidden pedestrian pixels. This indicates that failures in hiding pedestrian pixels on validation data are mostly due to overfitting to the training data; in fact the adversary succeeds in hiding nearly of all pedestrian pixels on the train set for any combination of number of training images and tile-size (not shown). The number of background pixels preserved typically decreases with increased score on hiding pedestrians. As this is also the case on training images, it is likely an underfitting or optimization issue which could be improved in the future by alternative regularization methods (other than periodic noise) or more sophisticated adversarial attacks. For the presented method and , a tile-size of achieves a good trade-off and is used in the remaining experiments.

Table 2 illustrates the influence of the maximum noise level . Values of below clearly correspond to an underfitting regime as the adversary is not capable of hiding all pedestrian pixels on the train data. For , failures of the adversary in hiding pedestrian pixels on validation data are mostly due to overfitting (see above). Additional capacity in the perturbation () is then used by the adversary to preserve the background even better but does not help in reducing overfitting. The influence of parameter , which allows controlling the trade-off between the primary and secondary objective, is investigated in Table 3: the larger , the more pedestrian pixels are hidden (but the background is preserved less well). Since the number of pedestrian pixels is considerably smaller than the number of background pixels, setting close to , e.g., presents a reasonable trade-off. In contrast, the unweighted loss with no ( no) fails since it focuses too much on preserving the background.

Generalizability

We have tested the effect of the universal perturbation generated for Cityscapes on CamVid [1] (without any fine-tuning on CamVid). An average of 78% of the pixels are transformed to the adversarial target for the static target segmentation. For dynamic target segmentation, an average of 84.5% pedestrian pixels are hidden and 79.6% of the background pixels are preserved. Thus, the perturbations generalize to a similar dataset with only a small decrease in performance. Moreover, we have evaluated the FCN’s static target universal perturbation on a PSPNet [29]. Adding the universal perturbation reduced the IoU between PSPNet’s predictions and the ground truth on Cityscapes from 75.8% to 8.8%. However, the IoU between the prediction and the adversarial target was also only 9.5%. In summary, the universal perturbation generalizes over networks as an untargeted attack but not as a targeted attack.

5 Conclusion and Outlook

We have proposed a method for generating universal adversarial perturbations that change the semantic segmentation of images in close to arbitrary ways: an adversary can achieve (approximately) the same desired static target segmentation for arbitrary input images that have nothing in common. Moreover, an adversary can blend out certain classes (like pedestrians) almost completely while leaving the rest of the class map nearly unchanged. These results emphasize the necessity of future work to address how machine learning can become more robust against (adversarial) perturbations

[31, 19, 12] and how adversarial attacks can be detected [15, 7]. This is especially important in safety- or security-critical applications. On the other hand, the presented method does not directly allow an adversarial attack in the physical world since it requires that the adversary is able to precisely control the digital representation of the scene. While first works have shown that adversarial attacks might be extended to the physical world [11]

and deceive face recognition systems

[22], a practical attack against, e.g., an automated driving or surveillance system has not been presented yet. Investigating whether such practical attacks are feasible presents an important direction for future work. Furthermore, investigating whether other architectures for semantic image segmentation [13, 30, 3, 28, 4] are less vulnerable to adversarial perturbations is equally important.

References

  • [1] G. J. Brostow, J. Fauqueur, and R. Cipolla. Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters, 30(2):88–97, Jan. 2009.
  • [2] N. Carlini and D. Wagner. Towards Evaluating the Robustness of Neural Networks. In arXiv:1608.04644, Aug. 2016.
  • [3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. In International Conference on Learning Representations (ICLR), 2015.
  • [4] L.-C. Chen, Y. Yang, J. Wang, W. Xu, and A. L. Yuille. Attention to scale: Scale-aware semantic image segmentation. In Computer Vision and Pattern Recognition (CVPR), 2016.
  • [5] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The cityscapes dataset for semantic urban scene understanding. In Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, USA, 2016.
  • [6] A. Fawzi, S.-M. Moosavi-Dezfooli, and P. Frossard. Robustness of classifiers: from adversarial to random noise. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 1632–1640. Curran Associates, Inc., 2016.
  • [7] R. Feinman, R. R. Curtin, S. Shintre, and A. B. Gardner. Detecting Adversarial Samples from Artifacts. In arXiv:1703.00410 [cs, stat], Mar. 2017.
  • [8] V. Fischer, M. C. Kumar, J. H. Metzen, and T. Brox. Adversarial Examples for Semantic Image Segmentation. In International Conference on Learning Representations (ICLR), Workshop, Mar. 2017.
  • [9] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR), 2015.
  • [10] K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. In Computer Vision and Pattern Recognition (CVPR), 2016.
  • [11] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial examples in the physical world. arXiv:1607.02533, July 2016.
  • [12] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial Machine Learning at Scale. In International Conference on Learning Representations (ICLR), 2017.
  • [13] Z. Liu, X. Li, P. Luo, C. C. Loy, and X. Tang. Semantic image segmentation via deep parsing network. In The IEEE International Conference on Computer Vision (ICCV), 2015.
  • [14] J. Long, E. Shelhamer, and T. Darrell. Fully Convolutional Networks for Semantic Segmentation. In Proceedings of Computer Vision and Pattern Recognition (CVPR), Boston, 2015.
  • [15] J. H. Metzen, T. Genewein, V. Fischer, and B. Bischoff. On Detecting Adversarial Perturbations. In International Conference on Learning Representations (ICLR), 2017.
  • [16] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard. Universal adversarial perturbations. In Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, USA, 2017.
  • [17] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. DeepFool: A simple and accurate method to fool deep neural networks. In Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, USA, 2016.
  • [18] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples. In arXiv:1602.02697, Feb. 2016.
  • [19] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks. In Symposium on Security & Privacy, pages 582–597, San Jose, CA, 2016.
  • [20] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems, 2015.
  • [21] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211–252, 2015.
  • [22] M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter. Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS ’16, pages 1528–1540, New York, NY, USA, 2016. ACM.
  • [23] A. Simonyan, Karen amd Zisserman. Very deep convolutional networks for large-scale image recognition. In The International Conference on Learning Representations (ICLR), 2015.
  • [24] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
  • [25] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf. DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 1701–1708, 2014.
  • [26] Z. Wu, C. Shen, and A. v. d. Hengel. High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks. In arXiv:1604.04339 [cs], Apr. 2016.
  • [27] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille. Adversarial Examples for Semantic Segmentation and Object Detection. Mar. 2017. arXiv: 1703.08603.
  • [28] F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In The International Conference on Learning Representations (ICLR), 2016.
  • [29] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia. Pyramid Scene Parsing Network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  • [30] S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr.

    Conditional random fields as recurrent neuronal networks.

    In The IEEE International Conference on Computer Vision (ICCV), 2015.
  • [31] S. Zheng, Y. Song, T. Leung, and I. Goodfellow. Improving the Robustness of Deep Neural Networks via Stability Training. In Computer Vision and Pattern Recognition CVPR, 2016.
  • [32] W. Zhu, X. Xiang, T. D. Tran, and X. Xie. Adversarial Deep Structural Networks for Mammographic Mass Segmentation. In arXiv:1612.05970 [cs], Dec. 2016. arXiv: 1612.05970.

Appendix A Supplementary material

In this supplementary material, we show additional higher-resolution illustrations of Figure 3 and Figure 6. We show three examples for a static target segmentation which correspond to the images where the approach worked best (Section B) and worst (Section C), and a randomly selected image (Section D). Similarly, we show three examples for a dynamic target segmentation which correspond to the images where the approach worked best (Section E) and worst (Section F), and a randomly selected image (Section G). Moreover, we also show the predictions of the network on the noise itself in Section H.

Appendix B Static Target Segmentation - Best Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target

(e) adv. example
(f) pred on adv. example

Appendix C Static Target Segmentation - Worst Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target

(e) adv. example
(f) pred on adv. example

Appendix D Static Target Segmentation - Random Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target

(e) adv. example
(f) pred on adv. example

Appendix E Dynamic Target Segmentation - Best Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target


(e) adv. example
(f) pred on adv. example

Appendix F Dynamic Target Segmentation - Worst Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target


(e) adv. example
(f) pred on adv. example

Appendix G Dynamic Target Segmentation - Random Example

(a) image (b) prediction on image

(c) universal noise (4x)
(d) adv. target


(e) adv. example
(f) pred on adv. example

Appendix H Network Predictions on Universal Perturbations

(a) Static universal perturbation (x4) (b) Prediction on (a)
(c) Dynamic universal perturbation (x4) (d) Prediction on (c)