Recently 3D data is often used as an input for Deep Neural Networks (DNNs) in many scenarios including healthcare, self-driving cars, drones, robotics, and many more[fernandes2021point, miotto2018deep]. These 3D data compared to 2D counterparts (which are projected form of 3D data) capture more information from the environment. Therefore, they can cause more accurate results as an output, specially in safety-critical applications such as self-driving cars. There are different representations of 3D data; including voxels, meshes, and point clouds. Since point clouds can be receipted directly from scanners, they can precisely capture shape details. Some DNNs like PointNet [qi2017pointnet], PointNet++ [qi2017pointnet1], and DGCNN [phan2018dgcnn] are designed to feed the order-invariant point clouds into models. Despite huge success in 3D deep learning models, they are vulnerable to adversarial examples, where adversarial examples are specific inputs intentionally designed to mislead the models. Although adversarial examples and robustness against them have been analyzed in-depth for 2D data [moosavi2016deepfool, naderi2021generating, carlini2017towards, goodfellow2015explaining, an2013feature, naderi2020scale, madry2019deep], they have been just started to be investigated in 3D space. In general, 2D and 3D adversarial attacks can be studied through different viewpoints.
The first view includes targeted [xiang2019generating, hamdi2020advpc, lee2020shapeadv, zhou2020lg, tsai2020robust, wen2020geometry] and untargetted [liu2019extending, arya2021adversarial, liu2020adversarial, yang2021adversarial, kim2021minimal, arya2021adversarial, zheng2019pointcloud]
attacks. In targeted attacks, the victim model classifies the data point to a specific target class, whereas in the untargetted scenario, the model may classify the point cloud to any class other than the original one. The second view comprises whitebox[xiang2019generating, liu2019extending, arya2021adversarial, liu2020adversarial, hamdi2020advpc, lee2020shapeadv, zhou2020lg, yang2021adversarial, ma2020efficient, tsai2020robust, kim2021minimal, zheng2019pointcloud] and blackbox attacks, where in the whitebox the attacker is aware of the entire details about the victim model’s parameters, but the blackbox attacker is only aware of the model output.
However, due to the inherent structure of 3D point clouds, several aspects are unique to 3D attacks.
Point shift (shifts a few points only and the number of points remains constant) [xiang2019generating, liu2019extending, hamdi2020advpc, lee2020shapeadv, tsai2020robust, kim2021minimal, liu2021imperceptible] vs. point add (adds a few points and increases the point numbers) [xiang2019generating, liu2020adversarial, yang2021adversarial, kim2021minimal, arya2021adversarial] vs. point drop (drops a few points and reduces the point numbers)[yang2021adversarial, wicker2019robustness, zheng2019pointcloud].
On-surface perturbation (perturbations are applied along the object surface.) [arya2021adversarial, hamdi2020advpc, lee2020shapeadv, zhou2020lg, tsai2020robust, wicker2019robustness, arya2021adversarial]
vs. out-surface perturbation (perturbations are applied outside the object surface; such as noise and outliers)[xiang2019generating, liu2019extending, liu2020adversarial, kim2021minimal].
Optimization-based (first the initial estimate of adversarial perturbation is considered as an optimization problem, then it is solved using some optimizers.)[xiang2019generating, hamdi2020advpc, lee2020shapeadv, tsai2020robust, kim2021minimal] vs. gradient-based (first gradients of the cost function corresponding to each input point are acquired. They are then used to acquire an adversarial perturbation such that the proposed attack has a more tendency towards being misclassified) [liu2019extending, liu2020adversarial, yang2021adversarial, wicker2019robustness].
Recently the interest in generating 3D adversarial examples has increased. Xiang et al. [xiang2019generating] proposed different approaches to performe 3D attacks by utilizing shifted points and adding several point clusters, small objects, or additional points. Liu et al. [liu2019adversarial] generate adversarial examples by adding sticks or line segments. Yang et al. [yang2021adversarial] utilize gradient-based attack methods to create point-add and point-drop attacks. Liu et al. [liu2021imperceptible]
improve the imperceptibility of adversarial attacks by shifting each point along its normal vector. The authors in[kim2021minimal, arya2021adversarial] generate adversarial point clouds by minimizing the number of manipulated points. Some methods [wen2020geometry, zhou2020lg, dai2021generating] manipulate the surface geometry to convert input point clouds to adversarial ones. Chengcheng et al. [ma2020efficient] proposed joint gradient-based attack to break the Statistical Outlier Removal (SOR) defense. Tsai et al. [tsai2020robust]zheng2019pointcloud] and Matthew et al. [wicker2019robustness] drop the most critical points based on saliency-based techniques.
To demonstrate the effectiveness of the proposed defense, some attacks [zheng2019pointcloud, xiang2019generating, tsai2020robust] are considered to cover the above categories. In this way, the robustness of the model on different types of attacks is studied.
To treat adversarial attacks, some defense methods are first introduced. In general, adversarial training and input restoration are two common methods to improve adversarial robustness. Adversarial training method considers a model training on a mixture of adversarial and clean examples. Input restoration method considers preprocessing on input data (before feeding to the model), to eliminate adversarial perturbations. In terms of input restoration, the Simple Random Sampling (SRS) [xiang2019generating], Statistical Outlier Removal (SOR) [zhou2019dup], adding Gaussian noise [yang2021adversarial], saliency map removal [liu2019extending], and Denoiser and UPsampler NETwork (DUP-Net) [zhou2019dup] are some defense techniques that enhance model robustness against 3D attacks by adding a preprocessing step before feeding input samples to victim models. Wu et al. [wu2020if] proposed the Implicit Function Defense (IF-Defense) to optimize and restore the input point coordinates by limiting the point perturbation and surface distortion. Some defense techniques [liu2021pointguard, dong2020self] directly modify the victim model structure to robust it against attacks. In terms of adversarial training, Liu et al. [liu2019extending] extend it to 3D point cloud. They train the model by point perturbation adversarial examples and the original point cloud. Liang et al. [liang2022pagn] adaptively generate perturbations by embedding the perturbation-injection module to the model, to generate perturbed features to improve the adversarial training performance. Sun et al. [sun2020adversarial]
proposed a type of pooling operation to enhance 3D adversarial training performance. Also, in another work, they use the self-supervised learning and adversarial training together as a defense method[sun2021improving].
There is an independent line of research in 2D space in which the authors have concentrated on understanding the inherent nature of adversarial examples to describe models’ behavior; which is essential to enhance models’ performance. However, due to the multilayer nonlinear structure of DNNs, obtaining an accurate description of them is close to impracticable. Ilyas et al. [ilyas2019adversarial] showed that adversarial perturbations are non-robust features for DNNs, and they lead to decrease the model’s accuracy. Some research tries to explain the adversarial examples by focusing on the frequency domain. Many of them show that adversarial perturbations concentrate more on high-frequency components [wang2020towards, yin2019fourier, ortiz2020hold]. However, the authors in [han2021rethinking] claim there are different frequency components in adversarial perturbations on various datasets. The frequency information modifies depending on the space that the object takes up in the 2D image. Also, there are some attacks [guo2018low, sharma2019effectiveness, duan2021advdrop] and defenses [lv2020frequency, song2021adversarial, wang2020high] are proposed from a Frequency Perspective to help improve the model’s robustness.
LV et al. [lv2020frequency]
used the low-frequency information image acquired using discrete Fourier transform (DFT) to suggest a defensive approach.
Song et al.
proposed a convolutional neural network with two branches, one of which uses a low-frequency information image obtained. They used the compression method with the nearest neighbor’s average for each pixel to get a low-frequency information image.
This paper, motivated by [lv2020frequency, song2021adversarial], examines 3D point clouds to find the effect of low-frequency information on adversarial model robustness. Unlike 2D images, DFT and compression do not work well for this purpose. We find spherical harmonic transformation is a better option in 3D space.
Only very recently, in 3D space, there is a parallel study [liu2022boosting] working on frequency domain in 3D adversarial attack methods. The work in [liu2022boosting] has suggested an adversarial attack based on the frequency domain when using DFT (graph-based). But our work proposes a defense method based on the frequency domain when using spherical harmonic transformation.
To the best of our knowledge, there is no study to analyze the adversarial examples from the perspective of the frequency domain on 3D data. This paper explores the properties of adversarial point perturbations in the frequency domain using spherical harmonic functions. Our analysis shows that there are more perturbations in the high-frequency components for many existing adversarial attacks. The analysis allows to understand the frequency space of adversarial examples. Furtheremore, the study on frequency space helps to improve model robustness.
To this end, the proposed defense method filters high-frequency input data components when fed as the training data. Since the model has never learned the high-frequency components of point clouds, it can be robust against adversarial examples that most of their perturbation is on high-frequency components. We also filter the high-frequency components of the adversarial examples before feeding them to the model to protect it from the vulnerability of possible adversarial examples.
The remaining parts of this paper are organized as follows. Section 2 introduces the background of the frequency domain, adversarial attack, and defenses on 3D data. The proposed 3D defense method is presented in Section 3. Experimental results are discussed in Section 4, and Section 5 concludes the paper.
In summary, the main contributions of this paper are given as follows:
Improving 3D point cloud transformation from the space domain to the frequency domain by taking advantage of the point cloud itself and not the corresponding mesh.
Analyzing 3D adversarial examples in terms of frequency domain.
Proposing two defense methods based on frequency domain for 3D adversarial examples that have better performance than baselines and state-of-the-art defenses.
Improving the robustness of models on standard inputs by training models using the low-frequency data information.
2 Background and Related Work
2.1 Point Clouds in Frequency Domain
Similar to the 2D shape processing, the transformation from the spatial domain to the frequency domain in 3D shapes give advantages for certain types of problems; such as data compression [compression2009], point cloud registration [huang2021robust, huang2020ridf], 3D models enhancement (robust to noise and outliers) [poulenard2019effective, zhang2020hypergraph, cohen2018spherical]
, Generative Adversarial Network (GAN) improvement[ramasinghe2020spectral], 3D shape reconstruction [shen20193d], and 3D-shape descriptors [vranic20013d]. There are different ways to transform from the spatial domain to the frequency domain.
A common way is using the Fourier transform [huang2021robust, huang2020ridf, compression2009, zhang2020hypergraph, vranic20013d, shen20193d]. For example, Robert et al. [compression2009] divide the 3D shape into N slices to convert it to the 2D space. They then apply the Fourier transform to each slice. Spherical harmonics [cohen2018spherical] are the extension of the Fourier series from the circle (one angular component) to the sphere (two angular components). They are a set of orthogonal basis functions defined on the surface of a sphere. Each function defined on the surface of a sphere can be written as a weighte sum of these spherical harmonics. It is, therefore, necessary to represent the 3D point cloud as a function defined on the surface of a sphere. In this regard, Ramsing et al. [ramasinghe2020spectral] place the 3D mesh on the unit sphere. Then, to sample the points, rays cast outward from the shape’s center. They consider the sum of the first and second mesh hit locations as the sample points.
2.2 Adversarial Examples and Defenses on 3D Point Cloud
Recently, adversarial attacks in 3D space have been investigated. The attacks desire to make a 3D classifier that is misclassify by shifting [xiang2019generating, liu2019adversarial, liu2019extending, yang2021adversarial, tsai2020robust, ma2020efficient], adding [xiang2019generating, yang2021adversarial, liu2019adversarial], or dropping [zheng2019pointcloud, wicker2019robustness] some points from the original point cloud. Following the pattern of 2D attacks that change the intensity of image pixels by -bounded criteria to generate adversarial perturbations, there are three popular choices for such criteria on 3D point clouds. These include -norm, Chamfer, and Hausdorff perturbation criteria. These criteria force the perturbations to shrink so that it is not noticeable to the human eye. Accordingly, Xiang et al. [xiang2019generating] proposed point generation attacks by shifting points with -norm and adding a limited number of points, clusters, or objects with Chamfer and Hausdorff perturbation criteria. It should be noted that all the attacks in [xiang2019generating] is designed as a targeted attack and out-surface perturbation. Later, there were many attacks proposed in [tsai2020robust, zhou2020lg, hamdi2020advpc] that, by adding different constraints, tried to generate stronger attacks. For example, Tsai et al. [tsai2020robust] proposed a targeted attack called KNN. The KNN attack adds a term to the loss function as a KNN distance constraint (when chamfer criteria is another constraint that exists to the loss function) so that the points are not too far from the surface and mostly are on-surface perturbation. All these attacks [tsai2020robust, xiang2019generating] use optimization-based approaches. On the contrary, some attacks focus on dropping points to generate adversarial attacks with gradient-based approaches. Zheng et al. [zheng2019pointcloud] drop the most critical points based on gradient-based techniques. As such, they generate an untargeted attack that is in the category of on-surface perturbation due to drop point.
Compared to 2D adversarial defense, not many techniques exist for the 3D point cloud adversarial defense.
Adversarial training is one of the most powerful defense techniques in the 2D defense techniques [tramer2020adaptive]. In standard training the model is trained only on standard point clouds. On the other hand, in adversarial training the model is trained with standard data and adversarial examples. Authors in [liu2019extending] train models with Shift-l2 attacks and authors in [liang2022pagn] train models with adaptive attacks. In that method, [liang2022pagn] add different types of attacks to the model. For example, an adaptive attack is designed to cover all types of attack. But, the proposed defense tends that models avoid outliers rather than see many outliers by removing high-frequency data information. In fact, the proposed defense considers the original and low-pass data while other adversarial training methods consider the original and high-pass data. Tables 1,2, and 3show that the proposed LPF2 has increased the model accuracy better over [liang2022pagn] and [liu2019extending]. Note that the bold numbers indicate the maximum performance in each column (attack).
Given that data quality is one of the most critical issues before performing any analysis, there are several forms of data preprocessing to remove the adversarial noise.
Typical defense techniques consider noise, outlier, or salient point removal. These defenses focus on add and shift adversarial examples and cannot perform well on drop attacks. Hence, recent defenses, such as IF-defense, try to improve model robustness on all type of attacks. Improving the robustness of models is a major challenge that has been studied in a variety of ways.
3 Proposed Method
In this section, the proposed method for converting 3D point clouds to the frequency domain is first introduced to remove high-frequency components in point clouds, effectively. Then, by utilizing the low-frequency point clouds, the proposed defense method to further improve the robustness of the models is explained.
3.1 Low-Frequency Point Cloud Information Extraction
There are two popular transformation to extract low-frequency point cloud information, namely the Fourier transform [dinesh2020point] and the spherical harmonics transform [cohen2018spherical]. Since the general purpose of this paper is to train the model with low-frequency information, both transformations are analyzed on model training. Figure 2 illustrates extracting the low-frequency information of a sample of ModelNet40 data with the Fourier and spherical transformation. As shown in Figure 2
, spherical harmonic transformation removes corners and preserves point clouds’ uniform distribution. In contrast, the Fourier transformation extracts a skeleton of point clouds very narrowly. Due to the fact that points in the low-frequency version of data based on the Fourier transform are concentrated in certain regions and are not uniformly distributed, they gained less model accuracy than the spherical harmonics. This means that the uniform distribution of points on the surface of the point cloud has a high effect and therrefore is essential in the training phase. In addition, the proposed method seeks to eliminate high frequencies where attack perturbations probably are more concentrated, such as corners. Therefore, it is concluded that the spherical harmonic transformation is more appropriate for the purpose of this paper.
Ramasinghe et al. [ramasinghe2020spectral] proposed a method based on spherical harmonics, which transforms 3D point clouds to the frequency domain. The proposed method takes an approach similar to that of [ramasinghe2020spectral], with the essential difference that the proposed method only takes advantage of the point cloud itself and not the corresponding mesh (which relaxes the requirement of having that mesh).
3.1.1 Projecting onto Unit Sphere
Coming to this point, the first step is to project the input point cloud onto the unit sphere, centered at its centroid. This projection is characterized as a non-negative function defined on an equiangular sampling grid on the sphere surface, in which and are the co-latitude and longitude, respectively. The value of at a particular grid point is defined as the radius of the point in the point cloud that its polar coordinates are the closest to . In this projection, several grid points might correspond to the same point in the point cloud. Also, there might be points with no grid points assigned to them. It is worth noting that the grid needs to be of sufficient resolution, so that it is able to capture the fine details of the input point cloud. After obtaining the projection on the sphere, spherical harmonics are used to transform the data to the frequency domain.
3.1.2 Spherical Harmonics
Analogous to Fourier series for functions on the unit circle, spherical harmonics are a set of complete and orthogonal basis functions for representing any function on the unit sphere . According to [ramasinghe2020spectral], any continuous function that satisfies a certain set of conditions can be written as
where are the spherical harmonics base functions of degree and order , and also , is corresponding coefficient of the base functions. In other words, the coefficients are the frequency domain representations of function . Ramasinghe et al. [ramasinghe2020spectral] provide more information on the definition of each and the formulation by which coefficients can be calculated for a given function . For the implementations of this paper, the python library "pyshtools" [wieczorek2018shtools] is used to perform spherical harmonics related operations.
3.1.3 Low-pass Filtering in Frequency Domain
After obtaining the point cloud projection onto the sphere, spherical harmonics are used to calculate the frequency domain coefficients. Each coefficient is then multiplied by a corresponding weight , which has the effect of low-pass filtering. Weights (
) come from a Gaussian distributionsuch that . The higher order coefficients, i.e., ’s with higher ’s, are multiplied by a smaller weight, resulting in low-pass filtering and thus diminishing noise and outliers. Our studies show that Gaussian filter performs better than box filters, (which are often used to cut-off frequencies). More details are given in Section 4.5
3.1.4 Reconstructing the filtered point cloud
The final step is to transform the coefficients back into spatial domain for retrieving the low-passed . For each spherical angle pair that have at least one point from the point cloud assigned to them during the initial projection, a point is generated based on the new value of . These newly generated points reconstruct the low-pass filtered point cloud. The size of the point cloud might be reduced after this process. The potential loss in the number of points can be compensated by randomly re-sampling existing points.
Note that in contrast to [ramasinghe2020spectral], point clouds themselves are the primary input here and not their corresponding mesh. This is crucial to the semantics of defending against adversarial attacks, as one is only given the point cloud data to perform the defense on.
3.2 Frequency-Based Analysis Attacks
The issue of defense is raised by introducing adversarial attacks. A better analysis of the adversarial attacks gives a better view of how to generate an attack. As a result, a more effective defense can be designed against it. Attacks retain the original appearance of the object and deceive the model by a few changes on some points such that they is not noticible be human eye. It is usual to suppose that adversarial perturbations affect more high-frequency components than low-frequency ones. This paper studies this assumption by analyzing the adversarial attacks in the frequency domain. For this purpose, the function measures the average dissimilarity between the spherical harmonics coefficients of the original point cloud and the adversarial one in the data test
where is the number of point clouds in the data test. and are coefficients corresponding to spherical harmonics base functions in an original point cloud and adversarial one , respectively. stands for the Spherical Harmonics Transform. For visualization to perform well under different adversarial perturbations, normalization is performed by dividing by the coefficients of original point clouds. According to the results obtained in Section 4.4, most adversarial perturbations occur at medium and high frequencies.
3.3 Adversarial Defense with Low-Frequency Point Cloud
Based on observations in the previous section, adversarial perturbations are found more in the mid- and high-frequency components of the adversarial attacks. Therefore, low-pass filtered versions of ModelNet40 data () are generated. In more detail, if represents the forward transform operator and ModelNet40 data indicated by , the is expressed as = , where denotes masking high-frequency components. Based on , two different defenses are proposed to improve the model’s robustness to adversarial examples.
In the first proposed method, called , models are trained with of original point cloud only and are tested with of adversarial examples. In this way, removing all high frequency components bypasses their usage by models. This filter causes the model training to focus only on the low-frequency information of original data. In result, the trained model can be more robust than perturbations and outliers. On the other hand, each adversarial example is removed from any high-frequency information before feeding to the model as test data, which can be similar to what the model was trained to do. In fact, the model can find the appropriate label for adversarial examples with a high probability.
In the second proposed method, called , models are trained by a mixture of original point cloud and its .
Adversarial training is one of the most powerful defenses in the 2D defense techniques, but it does not do well in 3D data. Due to the irregular structure of point clouds, it is very challenging to model adversarial points to eliminate their impact on defense. Injecting a particular type of attack into model training, unlike 2D data, cannot have much effect on the model’s robustness. In 2D data, the regular structure of the pixels improves the modeling of the adversarial distortions to some extent. Therefore, the issue of generalizing adversarial training to unseen attacks is more prominent in 3D data and can lead to instability of performance. To address this problem, focuses on data injection by removing some of their high-frequency components. In contrast, existing 3D adversarial training methods focus on data injection by adding these redundant features (high-frequency components) to the model during the training phase. That is why the proposed and methods boost model’s accuracy against 3D adversarial training.
4 Experimental Results
4.1 Datasets and 3D models
The experiments in this paper used aligned benchmark ModelNet40 [wu20153d] dataset for 3D object classification. The ModelNet40 dataset contains 40 object classes and 9,843 3D Computer-Aided Design (CAD) objects for training, and 2,468 3D CAD objects for testing. three state-of-the-art models, including PointNet [qi2017pointnet], PointNet ++ [qi2017pointnet1], and DGCNN [phan2018dgcnn] are adopted as victim classifiers that run on the ModelNet40 dataset. The models are trained with default settings.222The experiments were performed on a machine equipped with one NVIDIA Tesla P100-PCIe and 16 GB memory.
4.2 Attack Settings
The proposed method has been tested on un-targeted/targeted attacks. Attacks are fed to the pre-train victim model to evaluate the accuracy. Attacks are re-produced for each model for a fair comparison between LPF-Proposed and base defense methods according to IF-Defense [wu2020if] settings. A target class, which is not equal to the ground-truth class, has been randomly assigned to each ModelNet40 data test for targeted adversarial attacks. This assignment of the target classes was maintained unchanged in all attacks to remove the randomness effect. So there are 2468 attack pairs (victim, target) to measure the accuracy. For un-targeted attacks, all test objects contain 2,468 objects fed to the model to estimate the accuracy. Therefore, the basic adversarial examples including the point shifting (Shift-L2) [xiang2019generating], the point adding (Add-CD and Add-HD) [xiang2019generating], the kNN attack (Shift-kNN) [tsai2020robust], and the point dropping (Drop-100 and Drop-200) [zheng2019pointcloud] are employed for performance comparison purposes. When Shift-L2, Add-CD, Add-HD, and kNN attacks optimize a Carlini & Wagner () function with L2-norm, Chamfer, Hausdorff, and both Chamfer and K-nearest neighbors distance as a perturbation metric, respectively. The same as [wu2020if]
, a 10-step binary search with 500 iterations in each step is utilized to generate the Shift-L2, Add-CD, and Add-HD attacks. Also, 2500 iterations are used for Shift-KNN. Furthermore, for add points attacks (ADD-HD and ADD-CD), 512 points have been added to 1024 points in each point cloud. Drop-100 and Drop-200 attacks remove 100 and 200 points from 1024 points with the highest saliency scores, wherein every iteration, 5 points with the highest saliency scores are dropped. Then, a new saliency map is constructed for the remaining points. This process is repeated in next iterations to drop 100 and 200 points. Point dropping is under un-targeted settings and others are under targeted settings. All experiments in this paper were implemented using PyTorch.
4.3 Defense Settings
To verify the validity of the proposed defense method, this method has been compared with the SRS [yang2021adversarial], SOR [zhou2019dup], DUP-Net [zhou2019dup], If-Defense [wu2020if], Adv Training with Shift-L2 [liu2019extending], and Adv training with PAGN [liang2022pagn]
baselines. It is noteworthy that “adversarial” is simplified with “adv” in the text and tables. In SRS, the number of dropped random points is 500. In SOR, the hyperparameters are set to k = 2 and= 1.1. If-Defense suggests three different versions. This paper reports the results of IF-Defense based on optimization with ConvONEt implicit function networks, which is the best version of the If-Defense. The rest of the settings are in accordance with related reference papers.
|Clean||Shift-L2 [xiang2019generating]||Add-CD [xiang2019generating]||Add-HD [xiang2019generating]||Shift-KNN [tsai2020robust]||Drop-100 [zheng2019pointcloud]||Drop-200 [zheng2019pointcloud]|
|Adv Training (Shift-L2) [liu2019extending]||88.18%||43.28%||49.35%||53.47%||39.22%||70.23%||65.79%|
|Adv Training (PAGN) [liang2022pagn]||87.01%||84.83%||61.75%||64.35%||65.46%||66.29%||49.61%|
|Clean||Shift-L2 [xiang2019generating]||Add-CD [xiang2019generating]||Add-HD [xiang2019generating]||Shift-KNN [tsai2020robust]||Drop-100 [zheng2019pointcloud]||Drop-200 [zheng2019pointcloud]|
|Adv Training (Shift-L2) [liu2019extending]||89.14%||20.45%||13.01%||10.12%||9.05%||80.51%||66.98%|
|Clean||Shift-L2 [xiang2019generating]||Add-CD [xiang2019generating]||Add-HD [xiang2019generating]||Shift-KNN [tsai2020robust]||Drop-100 [zheng2019pointcloud]||Drop-200 [zheng2019pointcloud]|
|Adv Training (Shift-L2) [liu2019extending]||90.18%||13.21%||6.45%||6.41%||15.75%||75.42%||54.97%|
4.4 Adversarial attack analysis in frequency domain
This part analyzes the effect of adversarial attacks on frequency components by function that was introduced in Section 3.2. The distribution of perturbations point shifting (Shift-L2) [xiang2019generating], the point adding (Add-CD) [xiang2019generating], and the point dropping (Drop-100) [zheng2019pointcloud] in the frequency domain is visualized in Figure 3. The top vertex of the triangle shows the coefficient of low-frequency components, and as it moves down, the coefficient of high-frequency components are displayed. Based on the results, most of the adversarial perturbation is found in the mid- and high-frequency components of attacks. In other words, adversarial attacks deceive the model by modifying high-frequency components. According to Figure 3, the frequency components in the 20 rows above the triangle change slightly and most of changes happen below that. This is seen in almost every three adversarial attacks. This analysis can improve the learning phase of models. Once the model has also learned the low-frequency version of an object, the probability of changing the model’s decision is diminished by such perturbations.
4.5 Cut-off Frequency
Cut-off frequencies can be done in two ways of Box filter or Gaussian filter. Box filtering means, from one frequency onwards, all components are discarded (these frequencies would be set to zero). Gaussian filter means frequency components are weighted based on the Gaussian distribution (These frequencies would be high near zero and then decrease at higher frequencies according to the decay of the Gaussian distribution.) By setting the standard deviation, called, the Gaussian filter can control the cut-off frequencies. In other words, the higher gets, the higher the cut-off of high-frequencies occurs. In Figure 4, the results of two types of low-pass filtering is observed. The ripple in the flat region is due to the frequency cut-off with the Boxing filter. If a Gaussian filter is used instead of the Boxing filter, this ripple will be removed. The frequency response of the Box filter looks like a Box (or a rectangle). The impulse response of such a filter is the sinc function. That is why ripples are seen in the flat regions (flat surfaces of cubes seem to bend in after filtering). Note that Gaussian in the time domain maps to Gaussian in the frequency domain and does not suffer from ripples, so there is no bending-in in the flat surfaces.
4.6 Comparison of classification accuracy
Based on the observations in the previous section, adversarial perturbations are found more in the mid- and high-frequency components of the adversarial attacks. Therefore, low-pass filtered versions of ModelNet40 data () are generated, in which low-frequency components are retained by setting the standard deviation to = 20. The results of the two proposed defenses are examined using in the following.
The and are demonstrated in Tables 1,2, and 3 as and , respectively. In , models are trained with only and are tested with or (SOR+) of adversarial examples. In , a mixture of and its was injected into the model for training. Both methods are trained with data that do not have information from the high-frequency components. As such, the model is more robust to noise and outliers. Note that for drop attacks (Drop-100 and Drop-200), the SOR preprocessing is not needed due to the attack mechanism. In the rest of the attacks, the SOR preprocessing is applied to the point cloud as discussed in 4.6.
Table 1 indicates the point cloud classification accuracy of the proposed defenses and other defense strategies on various attacks. The classification accuracy of defenses is shown as a percentage of the correctly classified test point clouds. A higher classification accuracy in the victim model indicates that the defense is more effective.
According to the results shown in Table 1, the three SRS, SOR, and DUP-NET defenses work well on point add (add-CD, add-HD) and point shift (Shift-L2) attacks. Due to their mechanism, these defenses can eliminate out-surface perturbation points well.
On the other hand, KNN attack can keep the perturbation points almost on the surface and reduces the accuracy of these defenses. Nevertheless, accuracy of defenses still makes sense. The problem occurs when there is a point drop attack and no point to remove. In such attacks, the model accuracy decreases sharply; however, DUP-NET has improved the performance by combining SOR and UPSampler networks by around 4%. But, because upsampler increases points close to the input points, the defense cannot resist when the number of dropped points becomes too large. The proposed defense methods can improve the accuracies on add, shift, and drop attacks. On the other hand, If-Defense combines SOR and resamples points from the mesh. It also defines in another attempt, two loss functions that preserve point geometry and point distribution on the surface to improve the accuracy in drop and other attacks. Compared to IF-Defense, the proposed method improves the model accuracy by an average of about 2% on attacks listed in Table 1.
Adversarial training (with Shift-L2) trains the models with original training data and Shift-L2 attacks [liu2019extending]. Adversarial training (with PAGN), trains the models with original training data and adaptive attacks [liang2022pagn]. Adaptive attacks are designed to cover all types of attacks. In fact, these defenses draw the model’s attention to the high-frequency components of data (attacks). Note that the irregular structure of point clouds can lead to performance instability in such defenses.
Both and use for training. In , high-frequency information (which in most adversarial examples is attacked by attackers) is removed. For example, a drop attack typically generates adversarial examples by removing corner points and edges. On the other hand, and , during the training phase, have learned the data whose high-frequency information has been removed. Therefore, these attacks are more likely to be categorized correctly. As shown in Table 1, has higher accuracy than state-of-the-art defense ([wu2020if]), for about 4% in both Drop-100 and Drop-200 attacks.
In Table 1, in which the victim model is PointNet, the proposed method outperforms all defense approaches in all studies attacks except for the Shift-KNN attack for which the IF-Defense approach achieves a higher performance in about . Table 2 shows the same leading performance of the method in the PointNet++ model. However, in this case the proposed method performs better in the Shift-KNN attack, but has around lower accuracy in Shift-L2 attack compared to the state-of-the-art approach (i.e., IF-Defense.). It is valuable to note that the approach has higher accuracy in about 3%, 1%, 1%, 2%, and 2% in Add-CD, Add-HD, Shift-KNN, Drop100, and Drop200 attacks than the state-of-the-art approach respectively. Similarly, the results for the DGCNN model are also reported in Table 3. The proposed method outperforms the state-of-the-art approach in Shift-KNN, Drop100, and Drop200 by 3%, 5%, and 9%, respectively. The proposed method has an acceptable performance on Shift-L2, Add-CD, and Add-HD compared to other defense methods except for the IF-Defense approach. Note that the results of the first proposed method () are also reported in all tables, and they exceed most defense approaches. The best performance is achieved with the proposed defense, as described above.
As a preprocessing step for the add and shift point attacks, SOR is first applied to remove the outliers. Then, the low-frequency information of data is retained. The results show that combining two utterly different denoising mechanisms (SOR and ) helps to boost the model robustness. In Figure 5, the Add-CH attack has been applied to the original point cloud. Then, three different methods for removing outliers are tested. Firstly, with S = 20 removes most of the outliers, except for one point at the bottom and one point in the rightmost side of the object. Secondly, SOR method that is the best in dropping outliers, removes the outliers. As seen in this figure, except for the few points at the bottom of the object that are so close to the object, SOR removes the outliers, effectively. Finally, the combination of and SOR takes the advantage of both methods and therefore the resulting object has the least number of outliers.
4.7 Ablation study
The previous section 4.6 reported and results when low-frequency components were retained by setting the standard deviation to = 20. Given that generally performs better than , an ablation study has been performed on different values on . In fact, the effects of different low-pass frequencies on were examined by setting to 0, 4, 8, 12, 20, 50, and 100. Figure 6 shows the effect of different values on the six different adversarial attacks. In general, the accuracy starts to increase from = 0 and peaks at = 20 and then decreases. However, there are exceptions. For example, in drop-200 attack, the accuracy peaks at = 8 but again increases at = 20. The amount of that the model is trained on can significantly affect the model’s performance on the adversarial attacks. is taught by a mixture of ModelNet40 data and . depends on the value. If the amount of is too small (such as 0 or 4), the object’s appearance gets closer to the sphere (as seen in Figure 1). Also, most of the high frequencies’ information are not present in it. In such cases, objects from different classes are not different even in their appearance. For =20, both the object appearance and the amount of high frequencies’ information are acceptable in average. The more increases, the object’s appearance gets closer to the original point cloud. However, higher frequencies’ information in the objects undermines the model’s robustness to the adversarial examples.
Although the focus of this paper is on the adversarial robustness of the models, model training with can also improve the robustness of the models. It should be noted that adversarial robustness refers to improving model robustness on adversarial examples. But, the term robustness refers to improving the model’s performance on the original inputs. Figure 7 shows the standard accuracy in three different ways. In Method 1, the model is trained with (the original ModelNet40 dataset). In Method 2, the model is trained with (the low-pass version of the ModelNet40 dataset). In Method 3, the model is trained by a combination of and . Also, in Methods 2 and 3, the parameter S in is set to 0, 4, 8, 12, 20, 50, and 100. The accuracy of these three methods on original test data is shown in Figure 7. Note that all the three methods are evaluated on the original ModelNet40 test dataset. The only difference is in the training data. It is seen in this figure 7 that the accuracy of Method 3 with all S values and the accuracy of Method 2 with S values of 50 and 100 are higher than the standard accuracy, an evidence on the claim explained above.
In Method 2, the accuracy increases with increasing the parameter . At first (in the lower values), the appearance of the objects (as initially shown in Figure 1) is far from the original data and model training is done only with these objects. In these cases () and it makes sense for the model to predict lower accuracy than the standard accuracy (Method 1). As increases (), it is observed that the accuracy grows higher than Method 1. The appearance (refer to Figure 1) is more similar to the original one. Also, unlike Method 1, the training data still does not contain all high-frequency information, resulting in a better performance. In Method 3, adding to increases the accuracy in all different values. It seems that injecting low-frequency data () alongside can robust the model even against the original data. More interestingly, this accuracy peaks at . However, by increasing , the accuracy remains higher than Method 1. In fact, this analysis shows that removing tiny perturbations from training data in specific directions (high frequency) can lead to boost the model robustness.
In this paper, a novel perspective is used to analyzed adversarial perturbations where most of the perturbations are found in mid- and high-frequency components. Also, two defense methods for improving the model robustness from the perspective of the frequency domain were proposed. Based on the obtained results, removing high frequencies’ information from the training data can improve the model robustness in adversarial examples as well as original 3D point clouds. Experimental results showed that the proposed defenses can increase the model accuracy in comparison with the state-of-the-art defense method in six different adversarial attacks.
The authors would like to thank Professor Ivan V. Bajić and Dr. Chinthaka Dinesh for the helpful discussions.