Deep Neural Networks (DNNs) have achieved considerable accuracy in various tasks (especially in classification). However, several recent research has shown that DNNs are not reliable enough Carlini and Wagner (2017); Goodfellow et al. (2015); Madry et al. (2019); Naderi et al. (2021); Brown et al. (2018). Adding small perturbations in the input data can cause the DNN to misclassify them with high confidence Szegedy et al. (2014). This new input data is called an adversarial sample. It is a serious threat when it comes to safety-critical applications. Therefore, it is important to generate strong adversarial samples and study the DNN behavior against these adversarial samples. By considering the DNN behavior, the adversarial robustness of DNNs can be improved and more effective defenses can be constructed. Most attacks and defenses focus on 2D images Goodfellow et al. (2015); Carlini and Wagner (2017); Madry et al. (2019) They are still in the early stages on 3D data Liu et al. (2019b); Zhou et al. (2019); Yang et al. (2021). Paying attention to 3D data is interesting, because the world around us is a combination of 3D objects. In addition, 3D data has many applications in robotics, augmented reality, autopilot, and automatic driving. Thanks to the presence of 3D sensors, such as various types of 3D scanners, LiDARs, and RGB-D cameras (such as Kinect, RealSense, and Apple depth cameras), it is easier to capture 3D data. This paper proposes an untargeted 3D adversarial point cloud attack against point cloud classifiers; namely, PointNet, PointNet++, and DGCNN. The proposed attack, adds a few points while applying hard boundary constraints on the number of added points and on the point perturbation norms. By controlling the step-size, the generated adversarial sample yields to escape local optima and find the most appropriate attack. The rest of this paper is organized as follows. In Section 2, the related work is reviewed. The proposed 3D adversarial attack is introduced in Section 3. Experimental results are discussed in Section 4. Finally, Section 5 concludes the paper. In summary, the contributions of this work include:
Proposing an adversarial attack method to perform effective attacks while preserving the point cloud appearance by applying two hard boundary constraints on the number of modified points and on the point perturbation norms.
Proposing a learning rate scheduling algorithm to improve other existing methods in this setting.
Managing to generate highly successful attacks with a small number of steps to perform fast and subtle attacks.
2 Related Work
2.1 Deep Learning on 3D Data
There are three strategies for 3D object classification including volume-based Wu et al. (2015); Maturana and Scherer (2015), multi-view-based Su et al. (2015); Yang and Wang (2019), and point cloud-based Qi et al. (2017a, b); Wang et al. (2019). This research focuses on point cloud-based models. As a pioneering work, the PointNet Qi et al. (2017a)
can directly feed point clouds as its input. It achieves the features of each point independently and then aggregates them by max-pooling. It then extracts global features for 3D point cloud classification and segmentation tasks. An update of this work is the PointNet++. It improves the feature extraction through combined features from multiple scales in order to add locality to the PointNet. More recent work apply convolutions on neighborhood points to aggregate more local contextThomas et al. (2019); Hua et al. (2018); Wu et al. (2019); Li et al. (2018); Wang et al. (2019). For instance, DGCNN Wang et al. (2019) processes neighborhood points by applying EdgeConv to better capture local geometric structures of points and therefore achieves superior classification results. This paper uses the PointNet, PointNet++, and DGCNN architectures to evaluate the proposed attack in the case of 3D point cloud classification task.
2.2 Adversarial Point Clouds
Various studies have focused on adversarial attack on 3D point cloud classification. The adversarial attacks can be categorized into band-limited and unrestricted adversarial perturbations.
Band-limited approaches, apply the limitation on the perturbations in generated adversarial samples while preserving the point cloud appearance visually. Typical perturbation measurements include L2 norm, Chamfer distance, and Hausdorff distance. The band-limited attacks are divided into point addition, point shifting, and point dropping attacks. In terms of point addition, Xiang et al. Xiang et al. (2019) proposed three different targeted attacks by adding several point clusters, tiny objects, or extra points. These attacks optimize a Carlini & Wagner (C&W) function Carlini and Wagner (2017) and constraint point perturbation norm to push the added points towards the object surface. Yang et al. Yang et al. (2021) add a few points to the original point cloud based on the Fast Gradient Sign Method (FGSM) attack Goodfellow et al. (2015). They restrict both the number of added points and the point perturbation norm to generate imperceptible targeted adversarial samples. Liu et al. Liu et al. (2019a) add new points (sticks or line segments) into the original point cloud, where the sticks must arise from the object’s surface. The position of each stick onto the object’s surface and the number of points across the line segments are limited. In addition to generating point clouds by adding points into an original point cloud, both Zheng et al. Zheng et al. (2019) and Matthew et al. Wicker and Kwiatkowska (2019) iteratively drop points from the original point cloud to deceive the classifier. Also, Xiang et al. Xiang et al. (2019), Liu et al. Liu et al. (2019a, b), Yang et al. Yang et al. (2021), Tzungyu et al. Tsai et al. (2020), Hamdi et al. Hamdi et al. (2020), and Chengcheng et al. Ma et al. (2020) all propose adversarial attack based on point shifting methods. Most of those attacks extend 2D adversarial attacks Carlini and Wagner (2017); Goodfellow et al. (2015); Madry et al. (2019).
Another line of attacks focuses on unrestricted attacks, which are not limited to any distance criteria. These unlimited attacks do not necessarily look the same as the original point clouds. In other words, it is sufficient that the adversarial sample stays legitimate for the human eye but deceives the classifier. Applying isometry transformation on point cloud Zhao et al. (2020)
and using trained Generative Adversarial Network (GAN)Zhou et al. (2020) to generate adversarial sample are some research that has been proposed to unrestricted attacks. Since unlimited attacks do not visually preserve the point cloud appearance, they are not discussed in this paper.
There are typical adversarial defense methods including Statistical Outlier Removal (SOR)Zhou et al. (2019) and saliency map removal Liu et al. (2019b), which discard outlier and saliency points, respectively. Also, Zhou et al. Zhou et al. (2019) propose a denoiser and upsampler network (DUP-Net) structure as defenses for the 3D classification task.
This paper proposes an untargeted attack by a point addition method that imposes hard boundary constraints on the number of added points and on the point perturbation norms. To the best of our knowledge, all previous attacks train with a fixed learning rate until the objective function stagnates, but the proposed attack can escape local optima by controlling the learning rate. Furthermore, by imposing constraints on point perturbation norms and the number of added points it can find the most appearance preserving attacks.
3 Proposed Method
3.1 Problem Formulation
The objective is to generate an adversarial sample based on the original one, such that it deceives the model while retaining its appearance. Let be the original point cloud, where represents the coordinates of the th point. The adversarial sample is generated by adding point set to the original point cloud. The points in are denoted as adversarial points and the points in are denoted as original points. To deceive the model, the proposed method needs to maximize the model’s classification objective function . On the other hand, two hard boundary constraints are applied on the optimization problem to preserve the point cloud’s appearance. More concretely, the proposed method is designed to solve the following optimization problem
where and are hard boundaries on the perturbation norm and the number of added points, respectively. The function measures the dissimilarity between the adversarial sample and the original one. This function is chosen to be the Hausdorff distance defined by
where and are points from point clouds and , respectively. This function limits each point from to be in the neighborhood of its closest neighbor from . In other words, all the added points must be close to the surface of the original point cloud.
Although the applied constraints preserve the adversarial sample’s appearance, they make the search for the optimal much more difficult, especially in a first-order method. The Hausdorff distance constraint limits the movement of the added points which might lead the optimization process to get stuck in bad local maxima. Moreover, due to the low number of added points which leads to low search space dimension, the optimization problem landscape is very non-concave Engstrom et al. (2017). The proposed methods in the following sections overcome these problems to produce better results using first-order methods.
3.2 Variable Step-Size Attack
The main proposed method uses the Projected Gradient Descent (PGD) Madry et al. (2019) algorithm with high step-size at the beginning of the algorithm, to solve Equation 1. High step-size lets the added points to explore the whole surface of the point cloud efficiently, while giving them the possibility to escape from local maxima. To converge to the desired result, the step-size is gradually reduced throughout the algorithm.
The algorithm is summarized in Algorithm 1. Suppose that is the original point cloud. First, is initialized using the points of with the highest gradient norms. These points have the most effect on the classification objective function and are thus a good initialization. At each step, the points in take a step in the gradient direction, and in a random direction if their respective gradient norm is zero. At the end of each step, each point in is projected into the neighborhood of its respective nearest neighbor from . The step-size is reduced at each step to ensure the algorithm’s convergence. For further insights, the algorithm is demonstrated in Figure 1.
3.3 Variable Boundary Attack
An alternative way to overcome the problem of local maxima is to vary the parameter. In this method, the parameter is set high at the beginning of the algorithm and is reduced to the desired final value , during the algorithm. By having a relaxed constraint for the first steps of the algorithm, it is easier to find solutions with a high objective function value. By proceeding throughout the algorithm, the solutions found with higher values serve as good initialization points for lower values. This finally leads to a better solution for the desired value at the end of the algorithm, compared to using the PGD algorithm with a constant and parameter.
In this algorithm, compared to VSA, the hyperparametersand are replaced with initial boundary and step-size . Moreover, instead of , is updated using .
4 Experimental Results
In this section, the proposed method, which is denoted by VSA, is evaluated and compared with other methods to demonstrate its effectiveness. The experiments are carried out on three state-of-the-art point cloud processing architectures run on two benchmark datasets. The proposed method surpasses other state-of-the-art methods in attacking deep point cloud models when using a limited number of points. Moreover, it is shown that the proposed step-size scheduling algorithm can be adopted by existing methods to achieve higher results. The effectiveness of the proposed method against defense methods is also discussed in this section. Moreover, An ablation study is carried out to compare different variants of the proposed methods and to explore the effects of different hyperparameters. Finally, the generated samples of methods are visualized and compared in terms of perceptibility of adversarial points.
4.1 Experimental Setup
The state-of-the-art methods in the scope of the discussed problem, which include the Point-Attach Method (PAM) in Yang et al. (2021) and the Adversarial Sticks Method (ASM) in Liu et al. (2019a), are employed. For a fair comparison, the methods are chosen to be point addition methods which put hard boundary constraints on the perturbation norms and on the number of added points. In ASM, the farthest point sampling is avoided and new points are sampled onto the adversarial sticks, for it to be used as a point addition method.
4.1.2 Datasets and Architectures
The main experiments are carried out against three popular models; namely PointNet Qi et al. (2017a), PointNet++ Qi et al. (2017b), and DGCNN Wang et al. (2019). The benchmark datasets used for these experiments are ModelNet40 and ScanObjectNN. The ModelNet40 dataset is used for 3D CAD model classification. The training split of the dataset with samples is used to train the models and the test split with samples is used to evaluate the attack methods. On the other hand, the ScanObjectNN which is a real-world dataset consisting of indoor 3D objects is divided into a training split with samples and a test split with samples. The experiments are carried out on ModelNet40 against PointNet, if not mentioned otherwise. In all of the experiments the original point clouds have points and are normalized according to Qi et al. (2017a).
All the hyperparameters are initialized according to this section unless mentioned otherwise. For the VSA method, is set to and is set to . By this, if is low, the adversarial points spend more time exploring the surface of the point cloud which benefits them since they only cover a small portion of the surface and might need time to reach the optimal solution. Note that according to the observations, the adversarial points tend to distance from each other when proceeding towards the optimal solution. This is because for the studied models in this paper, it is observed that when two points get too close to each other, one overshadows the other’s contribution to the classification objective function. Since the points with the most impact on the classification objective function are chosen as the adversarial points’ initialization, they tend to be distanced from each other too. Therefore, if
is high, less time is needed to search the point cloud surface since the points already cover the majority of the point cloud surface, which makes them more probable to be close to the optimal solution.
For the Variable Boundary Attack (VBA) method, is set to and is set to . The should be low enough to propose a solution similar enough to the optimal one at the final , and it should be high enough to solve the problem of local maxima to a certain extent. This makes an appropriate initialization for . For , an initialization method similar to that of of VSA is chosen due to the reasons discussed in the previous paragraph. The number of steps is set to for both methods.
4.2 Obtained Results
All the attack success rates are reported in percentage. The attack success rates against the models trained on ModelNet40 and ScanObjectNN are reported in Tables 1 and 2, respectively. The reported results are against PointNet, PointNet++, and DGCNN in Table 1 and against PointNet in Table 2. The experiments were repeated for different pairs of constraint boundaries . Note that the nearest neighbor distance mean for the points in original point clouds (after normalization) is around . As such, makes the adversarial points stay near the point cloud surface. As reported in these tables, the proposed method outperforms other state-of-the-art methods by a large margin. It can be seen that PointNet and PointNet++ are very vulnerable against the proposed method. They almost misclassify every given sample when attacked by adding less than of the points, near the point cloud surface. In contrast, DGCNN performs much better against attack methods and is more challenging. Despite this, the proposed method manages to deceive this model of times. Moreover, as shown in Table 2, the proposed method manages to generate subtle adversarial samples with high accuracy on a real-world dataset. This shows the effectiveness of the proposed method in real-world settings.
|Model||Method||(0.05, 25)||(0.05, 100)||(0.1, 25)||(0.1, 100)|
|Method||(0.05, 25)||(0.05, 100)||(0.1, 25)||(0.1, 100)|
4.2.1 Adoption by Existing Methods
Due to the effectiveness of the proposed method, it can be adopted by other methods as well. To assess its effect, the step-size scheduling algorithm was used on PAM and ASM. For ASM, the learning rate was initialized with instead of and was reduced to throughout the algorithm. That method is denoted as ASM+. For PAM, the step-size is initialized with and is divided by at the end of each step. This version of PAM is denoted as PAM+.
The attack success rates of these methods and their enhanced versions are shown in Table 3. The results are reported for different pairs of constraint boundaries . As shown in this table, the proposed step-size scheduling algorithm is able to improve the existing algorithms for every pair of constraint boundaries. The improvements are more significant for , especially for the ASM algorithm where an improvement of is made with as parameters. The difference in improvement between ASM and PAM is most likely due to the usage of projection in ASM’s algorithm, which leads to a more effective search when paired with the step-size scheduling algorithm. Overall, this shows the impact of the proposed search strategy on other algorithms, in the scope of this problem.
4.2.2 Robustness Against Defense Methods
In this section, the proposed method is evaluated against statistical outlier removal (SOR) defense and salient point removal (SPR) defense. For the outlier removal defense, the
nearest neighbor average distance is calculated for each point. The points that have an average distance of greater than one standard deviation from the mean of this statistic are removed. For the salient point removal, thepoints with the highest saliencies are removed.
The attack success rates against these two defense methods are reported in Table 4. For each number of points , different values for were tested and the best result was reported. The SOR defense is more challenging when the number of points is low. This comes from the fact that the mean of the statistic calculated for the original point cloud does not change drastically when the adversarial points are added. Despite its challenges, the proposed method manages to evade the defense by adding the adversarial points very close to the point cloud surface with , while having a high classification objective function. This is why it outperforms other methods against SOR. However, it is not as effective as ASM when it comes to SPR. This is because in ASM, when the salient points which are usually on the head of the sticks are removed, there are other close adversarial points that will replace the head of the sticks and make a successful attack.
|Method||(0.05, 25)||(0.05, 100)||(0.1, 25)||(0.1, 100)|
|Defense||Method||(0.025, 25)||(0.05, 25)||(0.1, 25)||25||(0.025, 100)||(0.05, 100)||(0.1, 100)||100|
|Method||(0.05, 25)||(0.05, 100)||(0.05, 400)|
|VBA + VSA||37.4||76.4||94.5|
In this section, different variants of the proposed methods are compared to each other and different aspects of the proposed method are explored and evaluated. The simplest variant of the proposed method is the PGD algorithm, where the step-size is constant compared to VSA. In this method, is initialized according to VBA. A more complicated version of the proposed methods is a method comprising both of their ideas, denoted as VBA + VSA. In this method, the variable is scheduled according to VBA and is scheduled according to VSA. Moreover, a variant of VSA where is not set high (it is set to , , and for equal to , , and , respectively) is also considered. is set to for in that method. The attack success rates for different variants of the proposed methods are reported in Table 5. The experiments were carried out for different pairs of constraint boundaries. As shown in this table, VBA outperforms PGD by solving the problem of local maxima to a certain extent. Moreover, the methods improved by the proposed step-size scheduling algorithm outperform the other methods including to a large extent. It can be seen that needs to be set high for VSA to work effectively, especially when is set low. The improvement made by the step-size scheduling algorithm is due to a number of factors, like escaping improper local maxima, being able to explore the point cloud surface better, and converging to the desired result at the end of the algorithms. Between the algorithms that employ the proposed step-size scheduling, VBA + VSA performs slightly better. However, it is observed that VSA performs slightly better in Table 6 experiments when compared to VBA + VSA. Due to its slightly better performance and its simplicity, the VSA was chosen as the main proposed method.
Table 6 contains attack success rates for different values of and with . It can be seen that the proposed method performs very effectively, even when the number of steps is as low as . This makes the proposed method ideal for fast and subtle attacks. Note that a higher number of steps lets the adversarial points better explore the point cloud surface, though its impact slowly starts to decrease as the number of steps increases. Moreover, when the number of points is higher, the algorithm gets less affected by decreasing the number of steps, which is due to the effective initialization discussed in Section 4.1.3. To explore whether the local maxima problem is tackled or the improvements of VSA are merely due to the high number of steps and better exploration of the point cloud surface, the results in Table 6 are compared with Table 5. Consider where the local maxima problem is worse. By comparing the success rates of and in Table 6 to and of the method in Table 5 respectively, it can be seen that the respective success rates are higher. However, the respective traversed distance per points are lower. This shows that this improvement is due to solving the local maxima problem.
In this section, the perceptibility of the generated adversarial sample is explored. For this, the attack success rate is fixed on and are chosen for each of the methods to reach this success rate threshold. The generated samples are shown in Figure 2. Since there is a trade-off between and to increase success rate, the first column for each method contains samples with low and high and the second column contains samples with high and low .
As shown in Figure 2, the adversarial points are far less perceptible in the proposed method compared to existing methods. Moreover, it is more outlier free due to its ability to perform successful attacks with low .
This paper proposed a new attack method to generate effective adversarial attacks by adding a limited number of points to the point cloud surface. The method introduced an effective step-size scheduling algorithm to overcome the local maxima problem and to explore the point cloud surface efficiently. The results showed that in addition to achieving state-of-the-art results, it can be adopted by other existing methods to improve their results. It also showed that the proposed method performs well against the SOR defense which is more challenging when the number of added points is low. Overall, this shows that 3D deep learning models are vulnerable to subtle yet effective attacks. By these observations, future work could investigate different step-size scheduling algorithms and their effects on the performance of first order attacks.
- Unrestricted adversarial examples. arXiv preprint arXiv:1809.08352. Cited by: §1.
- Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39–57. Cited by: §1, §2.2.
- Exploring the landscape of spatial robustness. External Links: Cited by: §3.1.
- Explaining and harnessing adversarial examples. External Links: Cited by: §1, §2.2.
Advpc: transferable adversarial perturbations on 3d point clouds.
European Conference on Computer Vision, pp. 241–257. Cited by: §2.2.
Pointwise convolutional neural networks. In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 984–993. Cited by: §2.1.
- Pointcnn: convolution on x-transformed points. Advances in neural information processing systems 31, pp. 820–830. Cited by: §2.1.
- Adversarial point perturbations on 3d objects. arXiv e-prints, pp. arXiv–1908. Cited by: §2.2, §4.1.1.
- Extending adversarial attacks and defenses to deep 3d point cloud classifiers. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 2279–2283. Cited by: §1, §2.2, §2.2.
- Efficient joint gradient based attack against sor defense for 3d point cloud classification. In Proceedings of the 28th ACM International Conference on Multimedia, pp. 1819–1827. Cited by: §2.2.
- Towards deep learning models resistant to adversarial attacks. External Links: Cited by: §1, §2.2, §3.2.
- Voxnet: a 3d convolutional neural network for real-time object recognition. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922–928. Cited by: §2.1.
- Generating unrestricted adversarial examples via three parameters. External Links: Cited by: §1.
- Pointnet: deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660. Cited by: §2.1, §4.1.2.
- PointNet++: deep hierarchical feature learning on point sets in a metric space. External Links: Cited by: §2.1, §4.1.2.
- Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the IEEE international conference on computer vision, pp. 945–953. Cited by: §2.1.
- Intriguing properties of neural networks. External Links: Cited by: §1.
- Kpconv: flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6411–6420. Cited by: §2.1.
Robust adversarial objects against deep learning models.
Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, pp. 954–962. Cited by: §2.2.
- Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog) 38 (5), pp. 1–12. Cited by: §2.1, §4.1.2.
- Robustness of 3d deep learning in an adversarial setting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11767–11775. Cited by: §2.2.
- Pointconv: deep convolutional networks on 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9621–9630. Cited by: §2.1.
- 3d shapenets: a deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1912–1920. Cited by: §2.1.
- Generating 3d adversarial point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9136–9144. Cited by: §2.2.
- Adversarial attack and defense on point sets. External Links: Cited by: §1, §2.2, §4.1.1.
- Learning relationships for multi-view 3d object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7505–7514. Cited by: §2.1.
- On isometry robustness of deep 3d point cloud models under adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1201–1210. Cited by: §2.2.
- Pointcloud saliency maps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1598–1606. Cited by: §2.2.
- Lg-gan: label guided adversarial network for flexible targeted attack of point cloud based deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10356–10365. Cited by: §2.2.
- Dup-net: denoiser and upsampler network for 3d adversarial point clouds defense. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1961–1970. Cited by: §1, §2.2.