Universal Perturbation Attack Against Image Retrieval

12/03/2018 ∙ by Jie Li, et al. ∙ Xiamen University HUAWEI Technologies Co., Ltd. University of Oulu 0

Despite the remarkable success, deep learning models have shown to be vulnerable to the universal adversarial perturbation (UAP). The existing endeavors on UAP methods mainly focus on attacking the image classification models. Nevertheless, little attention has been paid to attacking image retrieval systems. In this paper, we make the first attempt for UAP attacking to deep feature based image retrieval. Concretely, attacking image retrieval is to make the retrieval system return more irrelevant images to the query at the top ranking list, whose key design is to corrupt the relationships among features. To this end, we propose a unified method to generate retrieval-based UAP to break the relationships between image features from point-wise, label-wise, and list-wise aspects. We further analyze the impact of the resizing operation in generating UAP, and thus provide a solution to attack high-performance retrieval systems with query resizing. We evaluate the proposed methods on four widely-used image retrieval datasets, i.e., Oxford5k and Paris6k with their revised versions, which lead to a significant performance drop in terms of different metrics, such as mAP, and mP@10. Finally, we test our attacking methods on the real-world visual search engine, i.e., Google Images, which demonstrates the potential of our methods.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 3

page 7

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Fig. 1: When added to natural images, a single universal perturbation which is invisible to human eyes in image space causes most images to shift a lot in the feature space and lose original neighbourhood relationships. The top is the perturbation and dots represent the features of images connected. (Best viewed in color.)

Image retrieval is a long-standing research topic in the computer vision society

[37]. Given a query image, it aims to find the relevant images from a dataset. Despite the extensive efforts in improving the search accuracy (e.g., new features like NetVLAD [2] and generalized-mean pooling [28]) or efficiency (e.g., new indexing like Hamming Embedding [12] or hashing [18, 35]), very little attention has been paid to the vulnerability of the state-of-the-art retrieval systems. In contrast, it has been well demonstrated in image classification that deep models are vulnerable to attacking. Various attacking techniques have been proposed, like model distilling [25, 26]

, transfer learning

[20], and gradient updating [1]. For instance, Szegedy et al. [33]

first proposed gradient-based learning using box-constrained L-BFGS to craft perturbation that can fool a neural network. Goodfellow

et al. [17]

proposed the fast gradient sign method, which linearizes the loss function and determines the perturbation with the sign of gradient. Contrary to previous methods computing perturbations on a single image, Moosavi

et al. [21] proposed an image-agnostic perturbation named universal adversarial perturbation (UAP) that can fool most images from a data distribution. UAP does not require complex optimization computation for perturbing a new datapoint and is suitable for online scenarios.

However, no existing work has touched the topic of attacking deep feature based image retrieval. In this paper, we aim to find a UAP for corrupting neighbourhood relationships in the feature space as depicted in Fig. 1, which fools the deep feature based image retrieval system. We argue that, there are three problems that prevent the existing UAPs methods from being directly used in image retrieval:

  • Most existing UAP based attacking methods are designed for image classification, whose goal is to change the probabilistic output of the deep models. Such methods require the training and testing sets to have a consistent data/label distribution. However, it is not satisfied in retrieval, since the training (reference images) and the testing (query images) can be independent.

  • The retrieval performance highly depends on whether the relationships (i.e., the pair-wise, label-wise and list-wise) among visual features are well captured, which differs from the UAP setting in classification. In classification, UAP [21]

    is generated via seeking a universal perturbation for a set of training data by sending this set to the decision boundary of the classifier. However, there is not a clear boundary for feature-feature similarity relationships in retrieval. Therefore, UAP can be insufficient to change the neighborhood relationships in the high-dimensional feature space. As shown in Table 

    III, the classical UAP cannot be directly applied in image retrieval.

  • Due to the fixed pooling layers and fully-connected layers in the classification model, the size of an input image is generally fixed, which makes the size of UAP fixed. However, the size of images being sent to deep feature extractor can vary, because the used global pooling layers [28]

    convert different-sized feature maps into a constant size vector. It also restricts the direct usage of the traditional UAP for retrieval.

Fig. 2: The pipeline of the proposed method. Perturbation is first resized to the same size of the input image which goes through random resizing layer with a random scale. Then both the resized input image and the sum of perturbation and input image are fed into CNN model to corrupt three relationships. Only gradient of the perturbation will be calculated during back propagation to update the perturbation.

To address the above challenges, in this paper we propose a novel method for crafting an image-agnostic universal adversarial perturbation to attack image retrieval. Concretely, we build a unified model to craft the UAP that breaks the neighborhood relationships among features by changing the input slightly. The unified model perturbs three relationships in feature-feature neighborhood structures, i.e., point-wise, label-wise, and list-wise. The point-wise approach exploits at a single feature at a time, which explicitly pushes away the features of adversarial examples from their original locations. The label-wise approach trains a classification model based on the pseudo-labels, which targets at corrupting the output of the classifier to break the semantic relationship. The list-wise approach constructs the triplet tuples based on the pseudo-labels, which aims at maximizing the hinge loss to break the ranking relationship. Eventually, the above three relationships are combined to a unified UAP, with a multi-task learning paradigm in model training for better attacking performance.

In addition, we propose a multi-scale random resizing scheme that allows UAP to be applied to input images at different resolutions, which shows better attacking performance than the methods with fixed scale experimentally. The pipeline of the proposed method is shown in Fig. 2.

Quantitatively, the proposed method leads to a large performance drop on standard image retrieval benchmarks, i.e., Oxford Buildings and Paris with their revised versions, tested on two CNN-based image representation [28, 29, 34] with three different CNN models [9, 16, 32]. The universal adversarial perturbation can drop the performance such as mAP and mP@10 by at least , which reveals the cutting-edge image retrieval systems is quite vulnerable to adversarial examples. Interestingly, we further evaluate our universal perturbation on the real-world image search engine, i.e., Google Images, which can also corrupt the output ranking list.

The rest of this paper is organized as follows: In Section 2, we briefly overview the related work of the proposed methods. Section 3 describes the proposed method based on perturbing three different relationships. The experimental results and analysis are given in Section 4. Finally, we conclude this paper in Section 5

2 Related Work

Visual Features for Retrieval. Image retrieval is a long-standing research topic in computer vision society [37]. Given a query image, the search engine retrieves related ones from a large set of reference images. A typical setting refers to extracting and comparing features between a query and references, such as global descriptors [24] and local descriptor aggregations [13, 30]. Nowadays, the most prominent retrieval methods are almost all based on CNNs [4, 5, 14, 10, 28, 29, 34]

. It mainly uses the pre-trained CNNs as global representation for images. To that effect, CNN models pre-trained with ImageNet

[6] (e.g. AlexNet [16], VGGNet [32] and ResNet [9]) are typically good enough to provide superior performance over hand-crafted features [5]. Babenko et.al. [5] further showed that fine-tuning the CNN models can further boost the retrieval performance. In this trend, many recent methods are proposed to construct trainable pooling layers for better feature representation. Representative methods include but not limited to, Maximum activations of convolutions (MAC) [29, 34], the weighted sum pooling (CroW) [14], and generalized-mean pooling (GeM) [28]. In this paper, we mainly consider two state-of-the-art pooling methods, i.e., MAC [29, 34] and GeM [28], with three different CNN models, i.e., AlexNet, VGGNet, and ResNet, to evaluate the performance of UAP attacking.

Adversarial Examples. Szegedy et al. [33] have demonstrated that neural networks can be fooled by adversarial example, which is a clean image being intentionally perturbed, e.g. by adding noise called adversarial perturbation that are quasi-imperceptible to human eyes. Subsequently, various methods have been proposed to generate such perturbations [8, 7, 22]. Simple methods such as FGSM [8] determine the perturbation with one-step gradient-based method. An iterative scheme is proposed in [17] to achieve better attacking performance via applying gradient ascent multiple times. Besides, complex approaches like [22] find perturbation from the perspective of classification boundary. However, these methods compute perturbations for each data point specifically and independently. More recently, Moosavi et al. [21] have shown that there exists universal adversarial perturbations (UAP), which aims to find an image-agnostic perturbation that can predict wrong labels for most natural images. UAP is a single adversarial noise that is offline trained and can adversarially perturb the corresponding outputs of a given model online. It is observed that perturbations crafted for specific models or training sets can fool other models and datasets [8, 33], referred as transfer attacking, which is widely adopted in black-box attack that no information about the model is known in advance.

3 The Proposed Method

In this section, we elaborate our universal perturbation learning for image retrieval.

3.1 Notations

For convenience, we denote universal perturbation by , feature vectors of the original image and the adversarial one by:

(1)

where is the function that outputs the feature vector through a CNN model, and are resizing processes for input image and universal perturbation respectively. Resizing processes will be detailedly elaborated in Section 3.3. The distance between two feature vectors and are characterized as a function .

3.2 The Proposed UAP

Our goal is to craft an image-agnostic perturbation that can corrupt the relationships between a query image and the references in the dataset, where the originally similar data should be dissimilar after adding a small perturbation. Our method seeks a universal perturbation , such that , for corrupting as much feature-feature similarity relationships in the data distribution as possible. We consider three relationships to be corrupted, i.e., point-wise, label-wise and list-wise. We then combine the three relationships into a unified UAP model.

Corrupting Point-wise Relationship. To corrupt the pairwise neighborhood relationship between a query and the references in the dataset, we resort to pushing the perturbed feature to the opposite-direction of the original feature, as to maximize the distance . Therefore, we maximize the following function:

(2)

where is the Euclidean Distance.

Corrupting Label-wise Relationship.

Except for the point-wise relationship, semantic relationship such as sharing the same category is also widely used in retrieval. Therefore, to further attack the retrieval system, we define a classifier, equipped by the cross-entropy loss function with FC layer and softmax layer. The distance function

in Eq. 2 can be re-defined as:

(3)

where is the output of classifier after softmax layer. Therefore, the new objective function is to corrupt the label-wise relationship via maximizing:

(4)
0:  Data set , parameters .
0:  Universal perturbation vector .
1:  Initialize
2:  repeat
3:     for each datapoint  do
4:        Randomly resize then resize perturbation accordingly
5:        Compute and update the gradients
6:        Update the perturbation by optimizing Eq. 10
7:        if  gets saturated then
8:           
9:        end if
10:     end for
11:  until convergence.
Algorithm 1 Universal Perturbation Learning for Attacking Image Retrieval.

Corrupting List-wise Relationship. Image retrieval can be further viewed as a ranking problem, from which perspective the list-wise relationship plays an important role [19]. To further attack image retrieval system, list-wise relationship also needs to be considered. Here we mainly corrupt the list-wise information with the triplet loss through the ordinal relation [18]. We adopt a landmark-based ordinal relation [3] that compares any query point to the landmarks111

Landmarks are generated via the K-means clustering.

. Mathematically, an ordered relation set can be written as:

(5)

We define as similar pairs of and that share the same cluster, and means the distance between clusters corresponding to samples and is the farthest. Therefore, a set of tuples belonging to the subset of can be recomputed. So, to attack the retrieval system, we maximize the traditional triplet loss as:

(6)

where is the function, and is the parameter representing the margin between the matched and unmatched samples.

Oxford5k Oxford5k Paris6k Paris6k
E M H E M H E M H E M H
Eval mAP mP@10 mAP mP@10 mDR
A-MAC O 57.11 45.23 32.96 10.43 57.25 55.43 15.36 65.64 63.99 46.93 20.06 88.00 91.29 58.29
S 39.55 31.37 23.01 5.89 43.09 42.17 9.56 48.11 43.07 33.41 12.89 65.86 67.43 38.86 30.72%
R 36.98 30.04 22.38 7.09 38.46 39.14 10.36 48.03 43.57 34.04 13.16 65.00 67.00 37.29 31.22%
P 29.54 23.18 17.33 3.97 33.09 33.27 7.51 43.14 38.96 30.46 10.99 54.29 56.71 29.57 44.20%
U 27.09 19.64 15.36 4.21 26.51 26.46 6.86 43.04 38.74 30.45 11.12 54.14 56.14 31.14 47.19%
A-GeM O 59.86 50.21 36.72 14.29 58.10 53.60 23.32 73.66 70.65 51.89 22.80 87.71 88.86 57.86
S 40.27 32.34 23.54 7.18 39.75 37.57 10.78 53.15 46.49 36.25 14.56 61.71 64.86 37.29 34.98%
R 29.58 23.70 18.70 6.58 27.91 26.60 8.89 49.79 43.64 34.89 12.96 55.43 57.43 27.29 45.87%
P 29.74 24.02 18.43 5.69 26.54 25.21 7.92 43.63 37.94 31.39 11.36 43.57 45.43 20.00 51.94%
U 27.48 23.78 17.95 5.58 26.91 24.32 8.32 41.64 36.43 29.95 11.27 40.43 42.71 22.57 53.51%
V-MAC O 81.45 75.07 57.15 29.96 78.60 78.33 45.57 88.31 86.39 69.60 44.97 93.57 96.86 84.71
S 42.81 36.77 29.53 15.15 38.46 38.89 21.89 35.82 31.05 27.23 11.56 30.14 30.86 18.00 58.78%
R 38.63 33.26 27.93 15.05 36.47 36.43 21.00 24.59 21.54 20.57 8.98 14.86 16.57 10.86 65.86%
P 36.48 31.53 26.10 14.09 30.59 31.00 19.86 25.52 23.07 21.42 9.64 18.43 20.00 12.43 66.83%
U 34.13 29.92 25.24 13.26 29.85 29.67 19.14 23.94 21.35 20.37 9.37 16.86 18.43 12.86 68.41%
V-GeM O 85.24 76.43 59.17 32.26 80.52 81.29 49.71 86.28 84.66 67.06 42.40 95.14 97.57 83.00
S 66.51 54.89 42.72 20.04 60.62 62.00 32.34 65.14 59.35 46.75 22.79 73.71 75.00 54.14 29.33%
R 46.46 41.60 32.70 15.59 38.68 39.57 22.57 35.91 32.19 28.67 12.90 23.86 24.43 17.14 58.70%
P 42.68 38.75 30.84 15.62 34.98 35.14 22.00 36.36 31.82 28.90 12.37 24.00 25.00 17.71 60.26%
U 40.83 35.10 28.52 14.04 33.22 33.86 20.36 36.10 32.84 28.53 11.66 27.29 28.00 17.14 61.56%
R-MAC O 81.69 73.85 56.14 29.80 78.33 79.86 46.57 83.55 81.56 63.91 39.06 93.52 96.71 79.57
S 47.75 39.80 30.24 14.22 47.87 46.86 23.00 50.62 45.84 36.77 16.22 57.29 59.57 34.00 45.39%
R 39.43 34.17 26.35 12.13 41.18 39.60 20.61 41.99 38.52 31.93 14.83 41.86 44.14 28.00 54.31%
P 34.88 32.12 25.07 13.02 36.62 35.14 19.14 34.14 31.19 26.48 12.34 33.86 34.57 22.29 60.11%
U 31.66 28.01 23.34 13.27 31.47 31.43 20.14 33.69 30.62 25.74 10.87 29.86 30.57 17.00 63.11%
R-GeM O 86.24 80.63 63.13 38.51 82.72 83.14 54.57 90.66 90.33 74.06 51.69 94.96 98.29 88.29
S 56.50 47.99 37.54 17.90 59.41 57.75 28.36 58.29 53.94 43.41 20.21 60.86 62.57 36.86 41.75%
R 48.15 39.82 31.36 15.48 46.23 45.00 22.86 51.90 48.46 38.51 16.71 55.86 57.29 32.86 50.25%
P 35.01 31.19 25.74 14.40 36.18 35.71 22.14 38.22 35.80 29.60 12.35 31.86 33.14 19.43 62.88%
U 32.89 29.57 24.58 13.76 33.24 33.14 20.43 31.62 29.78 25.06 10.21 23.43 24.57 14.71 67.52%
TABLE I: The attacking results on four evaluated datasets with six CNN models. Lower mAP and mP@10 and higher mDR(mean dropping rate) mean better performance. We evaluate the performance of attacking with three different relationships and the unified model on six retrieval models. P: Point-wise, S: Label-wise, R: List-wise, U: Unified model; O: Original Results.

The Unified Model. The above three relationships reflect different neighborhood structures to characterize image retrieval. The point-wise method directly considers the similarity in feature space, the label-wise method mainly focuses on the semantic neighborhood relationship, and the list-wise method represents the ordinal relationship by constructing the triplet-tuples. However, it is not guaranteed that a single perturbation can corrupt all these neighborhood relationships once at a time.

To further improve the generality and robustness of UAP attacking, we resort to a multi-task learning paradigm to construct a unified model to craft an integrated UAP for image retrieval. Inspired by [15], we consider a unified model combining these three objective functions. The overall objective function of this unified model can be written as:

(7)

where is the relative weight of the loss , which can be directly updated via stochastic gradient descend. Maximizing Eq. 7 crafts the universal perturbation to corrupt point-wise, label-wise, and list-wise relationships jointly. The detailed algorithm is provided in Alg. 1.

3.3 Random Resizing

Unlike classification models, where input images are cropped and padded to a fixed size, retrieval model can accept inputs at different scales. Resizing not only affects the retrieval performance, but also influences the attacking quality. Notably, resizing itself is an effective means to defense attacking, as mentioned in

[36]. Therefore, different image sizes increase the difficulty of universal perturbation.

To make the proposed universal perturbation suitable for different scales, a random resizing process is employed, which resizes the original input image with size to a new image with random size . Note that, along with is within a specific range, and should be within a reasonably small range to prevent image distortion. Then, the UAP is resized to a new perturbation with the same size as to be added to the input image.

Inspired by [11], the function can be written as:

(8)

where the and denote the value of and at location and in channel , respectively. For back-propagation, the partial derivative can be calculated as:

(9)
A-MAC A-GeM V-MAC V-GeM R-MAC R-GeM
A-MAC 47.19 26.91 17.69 16.4 13.99 15.98
A-GeM 30.59 53.07 14.78 16.81 13.17 16.02
V-MAC 15.93 13.44 68.41 64.24 22.07 20.45
V-GeM 13.03 12.40 61.85 61.56 19.22 21.25
R-MAC 11.16 8.83 19.56 21.88 63.11 67.78
R-GeM 12.88 11.84 20.31 22.34 59.37 67.52
TABLE II: Evaluation results about transfer attack. The mean dropping rates are reported here, where a larger number means better attacking performance. The universal adversarial perturbation are generated via the unified model in Eq. 7.

3.4 The Optimization

Since the gradient of Eq. 2, 4, 6, and 7 can be easily got, we use the gradient descent with momentum [7] to update the perturbation vector at the -th iteration:

(10)

where is the momentum of the -th iteration and is the learning rate. The clipping operation that ensures constraint may invalidate updates after reaches a constraint. We tackle this issue following [23], which rescales to half when perturbation gets saturated.

4 Experiments

In this section, we present quantitative results and analyses to evaluate the proposed attacking schemes. We train our universal perturbation on the Structure-of-Motion Reconstruction dataset. Two recent CNN-based image descriptors (i.e., MAC [29, 34] and GeM [28]) with three different CNN basic models (e.g. AlexNet [16], VGGNet [32] and ResNet [9]) are used, forming six CNN models that are trained on the Structure-of-Motion Reconstruction dataset. We use Oxford5k and Paris6k with their revised versions to evaluate the attacking performance.

Training datasets. The SfM dataset [31] consists of million images downloaded from Flickr. It contains two large scale-training sets named SfM-30k and SfM-120k, respectively. We utilize K-Means clustering on 6,403 validation images from SfM-30k to obtain the list-wise relationship, and then use the clustering index as pseudo-label to train a classification model to obtain the label-wise relationship. Our universal perturbation is trained on 1,691 query images from the SfM-30k.

Evaluational Datasets. The Oxford5k dataset [27] consists of 5,062 images and the collection has been manually annotated to generate a comprehensive ground truth for 11 different landmarks, each represented by 5 possible queries. Similar to Oxford5k, the Paris6k dataset [27] consists of 6,412 images with 55 queries. Recently, Radenovi´c et al. [27] have revisited these two datasets to revise the annotation errors, the size of the dataset, and the level of challenge. The Revisited Oxford5k and Revisited Paris6k datasets are referred as Oxford5k and Paris6k respectively. We report our results on both old and revisited datasets, respectively.

Visual Features. For CNN-based image representation, we use AlexNet (A) [16], VGG-16 (V) [32] and ResNet101 (R) [9] pre-trained on ImageNet [6] as our base models to fine-tune the CNN models on the SfM-120k dataset. For the fine-tuned features, we consider two cutting-edge features, i.e., the generalized mean-pooling (GeM) [28]

and the max-pooling (MAC)

[29, 34]. As a result, we obtain a total of 6 features to evaluate the attacking performance, which are termed as A-GeM, V-GeM, R-GeM and A-MAC, V-MAC and R-MAC.

Evaluation Metrics.

To measure the performance of universal perturbation for retrieval, we mainly consider three evaluation metrics

mAP ,mP@10, and the fooling rate. Unlike classification, the fooling rate of top-1 label prediction cannot be computed directly for image retrieval. Therefore, we define a new metric to evaluate the fooling rate for retrieval, which is termed the dropping rate (DR). DR is defined as:

(11)

where is an adversarial example of original example , and is the metric used in retrieval such as mAP. Dropping rate characterizes attacking performance by measuring how much retrieval performance degrades. It is supposed to be within a range for a valid attacking, while or a negative value indicates the attacking is totally failed. The higher dropping rate, the more successful the attack is.

Oxford5K Paris6K mDR
V-MAC O 81.45 88.31
UAP 79.78 81.06 7.02%
Ours 34.13 23.94 65.49%
R-MAC O 81.69 83.55
UAP 77.95 83.79 -0.03%
Ours 31.66 33.69 60.46%
TABLE III: Comparison between our method with classical UAP [21] on Oxford5k and Paris6k with VGG16-MAC and Resnet101-MAC. O: The Original Retrieval Results.

4.1 Results of UAP Attack

We evaluate the performance of six state-of-the-art deep visual representations to universal perturbation using Alg. 1, the quantitative results of mean fooling rates, mAP and mP@10 are shown in Table I and Table II.

Table I shows the attacking performance on all four evaluation datasets. Clearly, for all deep visual features, all kinds of universal perturbations achieve very high dropping rates on the validation set. Most of them achieve a dropping rate of more than , which means that most relevant images will not be returned to the front of the ranking list. Specifically, the universal perturbations computed for V-MAC and R-GeM achieve nearly dropping rate.

Through the results in Table I, these three relationships can help to craft a robust universal adversarial perturbation. Notably, point-wise relationship plays an important role in generating universal perturbations, most of which can achieve the second place in our experiments. Moreover, the best performance is achieved by the unified model in Eq. 7, which has an averaged improvement when comparing to the second best ones (e.g., Point-wise and List-wise). We conclude that the point-wise, label-wise, and list-wise relationships are all suitable for universal perturbation generation, while their combination further enhances the attacking quality.

(a) Oxford5k (b) Paris6k
Fig. 3: The visualization results on Paris6K for ResNet101-GeM. All the images in red box are the queries, and the remaining pictures are sorted from left to right. The 5 rows show the retrieval results by using the original images and perturbed images via the point-wise relationship, label-wise relationship, list-wise relationship and unified model, respectively. (Best viewed in color.)

4.2 Results of Transfer Attack

As mentioned in Section 2, transfer attack is to fool models or dataset with a perturbation generated on another model or dataset. Table II shows the results about the transfer attack across different visual features, in which we report the mean dropping rate (mDR) calculated on all four evaluation datasets. The universal perturbation is trained on one architecture (e.g., V-GeM) , whose attack ability is evaluated to fool the retrieval system based on the other deep features (e.g., R-MAC or V-MAC222We consider that different CNN architecture with the same pooling method as different features.). Note that, all the perturbations are trained with the unified model in Eq. 7 on the same training set. Each row in Table II shows the mean dropping rates for perturbation crafted by a given feature, and each column shows the transfer dropping rates on the target feature.

Resize Scale Range A-GeM V-GeM
21.82% 1590%
7.69% 6.74%
53.06% 61.53%
53.07% 61.56%
50.84% 60.78%
51.94% 61.34%
TABLE IV: The effect of resizing in attacking.

Interesting observations are reported: The universal perturbations generated from the same network architecture can be transferred well to different but related models with different pooling methods. For some architectures, the universal perturbations generalize very well across other architectures, which even achieve higher dropping rate. This phenomenon is mainly due to different pooling methods after the same network structure. For example, we train the perturbation on R-MAC and attack the retrieval system on R-GeM, by which the dropping rate is better than the diagonal value obtained for dedicated optimizing for R-GeM. Moreover, with different basic CNN architectures, the MAC pooling always has better attacking performance than GeM. Therefore, we conclude that MAC pooling is more robust to GeM, although GeM has better retrieval performance.

Fig. 4: Universal adversarial perturbations crafted by the proposed method for multiple architectures trained on SfM. Corresponding features and deep architectures are mentioned below each image. (Best viewed in color.)

4.3 On the Effect of Resizing

We further train the classical universal perturbation [21] on the ImageNet validation set, and evaluate the effect of resizing in retrieval. As mentioned before, when extracting image features, we need to resize the input image. For classical UAP, we resize the existing UAP from to the target size of the input image. As shown in Table III, although the classical UAP works reasonably well for transfer attack on cross-data task, it can not be used directly for retrieval. To explain, resizing is an effective defense that not only protects against generalized perturbations, but also improves the model robustness to some extent.

Above results further inspire us to investigate resizing when attacking image retrieval. More quantitative results are shown in Table IV. We first set the resizing scale to a fixed , which has the same scale to the SfM training dataset. The dropping rate for Alexnet-GeM is about which is significantly lower than our multi-scale random resizing method. Moreover, we also report the results that the perturbation and the training images are both resized to a fixed scale of . We observe that it has almost no attacking capability to image retrieval, since most query images are at a smaller scale than this scale. Finally, we evaluate the influence of the range for our multi-scale random resizing. We observe that the proposed method is not very sensitive to different image sizes. Instead, a larger interval helps to improve attacking performance.

4.4 Visualization

Fig. 3 shows the retrieval results of ResNet-GeM features from the Paris6K evaluation set, along with their ranking results for the original images. In the first row, all the relevant images are retrieved by the ResNet-GeM features. However, in the following four lines, none of the images are related to the query when adding the universal perturbation. In details, to attack the label-wise relationship, the model aims to learn the perturbation to push the original image to the other categories. In the second row, we observe that the top 5 retrieved images are most relevant to the category of oil painting, instead of the true category of building. When multiple relationships are considered, the difference is even more significant among the top-5 images, but all of which are not relevant to the query.

We then visualize the perturbations that are trained from different basic CNN models in Fig. 4. The first three perturbations generated from different networks show large difference, while the last four perturbations trained from the same network with different objective functions share similar appearances.

At last, we show the attacking results on a real-world image retrieval system, i.e., Google Image in Fig. 5

. The even rows show the perturbed images along with the retrieved images and predicted keywords provided by Google Image, which are completely different from the original ones at the odd rows. For example, the original input is categorized to the Ashmolean museum, while the adversarial example changes to a palace. The attacking results have demonstrated that the proposed method can generate universal perturbation to fool the real-world search engine.

Fig. 5: Retrieval results on Google Images. The odd rows and even rows show the images retrieved by original query images and the corrupted ones by our universal perturbation, respectively. The predicted keywords via Google Image are also given.

5 Conclution

In this paper, we are the first to propose a set of universal attacking methods against image retrieval. We mainly focus on attacking the point-wise, label-wise, and list-wise neighborhood relationships. Towards more general and robust attacking, a multi-task learning scheme is further proposed to ensemble these three relationships to craft a better universal perturbation. We further analyze the impact of resizing operations in generating universal perturbation in details. Correspondingly, a multi-scale random resizing method is employed, which further improves the success rate of the above attacking schemes. We evaluate our proposed method on widely-used image retrieval datasets, i.e., Oxford5k, and Paris6K, in which our method leads to a large drop in a serial of retrieval models. Finally, we also attack the real-world system, i.e., Google Images, which further demonstrates the efficacy of our methods. Last but not least, our work can therefore be used as an inspiration in designing more robust and secure retrieval models against the proposed attacking.

6 Acknowledge

This work is supported by the National Key R&D Program (No. 2017YFC0113000, and No. 2016YFB1001503), Nature Science Foundation of China (No. U1705262, No. 61772443, and No. 61572410), Post Doctoral Innovative Talent Support Program under Grant BX201600094, China Post-Doctoral Science Foundation under Grant 2017M612134, Scientific Research Project of National Language Committee of China (Grant No. YB135-49), and Nature Science Foundation of Fujian Province, China (No. 2017J01125 and No. 2018J01106).

References

  • [1] N. Akhtar and A. S. Mian. Threat of Adversarial Attacks on Deep Learning in Computer Vision - A Survey. IEEE Access, 2018.
  • [2] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. Netvlad: Cnn architecture for weakly supervised place recognition. In

    Computer Vision and Pattern Recognition (CVPR)

    , 2016.
  • [3] E. Arias-Castro. Some theory for ordinal embedding. arXiv preprint arXiv:1501.02861, 2015.
  • [4] A. Babenko and V. Lempitsky. Aggregating local deep features for image retrieval. In International Conference on Computer Vision (ICCV), 2015.
  • [5] A. Babenko, A. Slesarev, A. Chigorin, and V. Lempitsky. Neural codes for image retrieval. In European Conference on Computer Vision (ECCV), 2014.
  • [6] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition (CVPR), 2009.
  • [7] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li. Boosting adversarial attacks with momentum. In Computer Vision and Pattern Recognition (CVPR), 2018.
  • [8] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples (2014). In International Conference on Learning Representations (ICLR), 2015.
  • [9] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Computer Vision and Pattern Recognition (CVPR), 2016.
  • [10] N. Hyeonwoo, A. Andre, S. Jack, W. Tobias, and H. Bohyung. Large-scale image retrieval with attentive deep local features. In International Conference on Computer Vision (ICCV), 2017.
  • [11] M. Jaderberg, K. Simonyan, A. Zisserman, et al. Spatial transformer networks. In Neural Information Processing Systems (NIPS), 2015.
  • [12] H. Jegou, M. Douze, and C. Schmid. Hamming embedding and weak geometric consistency for large scale image search. In European Conference on Computer Vision (ECCV), 2008.
  • [13] H. Jégou, M. Douze, C. Schmid, and P. Pérez. Aggregating local descriptors into a compact image representation. In Computer Vision and Pattern Recognition (CVPR), 2010.
  • [14] Y. Kalantidis, C. Mellina, and S. Osindero. Cross-dimensional weighting for aggregated deep convolutional features. In European Conference on Computer Vision (ECCV), 2016.
  • [15] A. Kendall, Y. Gal, and R. Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Computer Vision and Pattern Recognition (CVPR), 2018.
  • [16] A. Krizhevsky, I. Sutskever, and G. E. Hinton.

    Imagenet classification with deep convolutional neural networks.

    In Neural Information Processing Systems (NIPS), 2012.
  • [17] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533, 2016.
  • [18] H. Liu, R. Ji, J. Wang, and C. Shen. Ordinal constraint binary coding for approximate nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence (PAMI), 2018.
  • [19] T.-Y. Liu et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 2009.
  • [20] Y. Liu, X. Chen, C. Liu, and D. Song. Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770, 2016.
  • [21] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard. Universal Adversarial Perturbations. In Computer Vision and Pattern Recognition (CVPR), 2017.
  • [22] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Computer Vision and Pattern Recognition (CVPR), 2016.
  • [23] K. R. Mopuri, A. Ganeshan, and R. V. Babu. Generalizable data-free objective for crafting universal adversarial perturbations. IEEE transactions on pattern analysis and machine intelligence (PAMI), 2018.
  • [24] A. Oliva and A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision (IJCV), 2001.
  • [25] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami.

    Practical Black-Box Attacks against Machine Learning.

    In ACM, 2017.
  • [26] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy (SP), 2016.
  • [27] F. Radenovic, A. Iscen, G. Tolias, Y. Avrithis, and O. Chum. Revisiting oxford and paris: Large-scale image retrieval benchmarking. In Computer Vision and Pattern Recognition (CVPR), 2018.
  • [28] F. Radenović, G. Tolias, and O. Chum. Fine-tuning cnn image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence (PAMI), 2018.
  • [29] A. S. Razavian, J. Sullivan, S. Carlsson, and A. Maki. Visual instance retrieval with deep convolutional networks. ITE Trans. MTA, 2016.
  • [30] J. Sánchez, F. Perronnin, T. Mensink, and J. Verbeek. Image classification with the fisher vector: Theory and practice. International Journal of Computer Vision (IJCV), 2013.
  • [31] J. L. Schonberger, F. Radenovic, O. Chum, and J.-M. Frahm. From single image query to detailed 3d reconstruction. In Computer Vision and Pattern Recognition (CVPR), 2015.
  • [32] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  • [33] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
  • [34] G. Tolias, R. Sicre, and H. Jégou. Particular object retrieval with integral max-pooling of cnn activations. In International Conference on Learning Representations (ICLR), 2016.
  • [35] J. Wang, T. Zhang, N. Sebe, H. T. Shen, et al. A survey on learning to hash. IEEE transactions on pattern analysis and machine intelligence (PAMI), 2018.
  • [36] C. Xie, J. Wang, Z. Zhang, Z. Ren, and A. Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations (ICLR), 2018.
  • [37] L. Zheng, Y. Yang, and Q. Tian. Sift meets cnn: A decade survey of instance retrieval. IEEE transactions on pattern analysis and machine intelligence (PAMI), 2018.