Abstract
Many efforts have been made on developing adversarial attack methods on point clouds. However, without fully considering the geometric property of point clouds, existing methods tend to produce clearly visible outliers. In this paper, we propose a novel NormalAttack framework towards imperceptible adversarial attacks on point clouds. First, we enforce the perturbation to be concentrated along normals to deform the underlying surface of 3D point clouds, such that tiny perturbation can make the shape deformed for better attack performance. Second, we guide the perturbation to be located more on regions with larger curvature, such that better imperceptibility is achieved. Extensive experiments on three representative networks, e.g., PointNet++, DGCNN, and PointConv, validate the effectiveness of NormalAttack and its superiority to state-of-the-art methods.
1. Introduction
With the development and popularity of deep neural networks (DNNs) [1], their performance on 3D point cloud perception has been significantly improved [2–5]. However, DNNs are reported to be vulnerable to adversarial attacks [6], in which case imperceptible modifications on input samples can lead to erroneous predictions of victim models. Therefore, point cloud perception solutions based on DNNs suffer from the hidden security risk of adversarial attacks, hindering their deployment in safety-critical applications, e.g., autonomous driving [7], 3D object recognition [8, 9], and grasp planning of robotics [10].
In the last few years, many efforts have been made on developing adversarial attack algorithms for DNNs in the field of point clouds. By utilizing the unstructured nature of point clouds, many early research performed attack by adding adversarial points, clusters, and objects [11], or dropping a small set of salient points [12]. To learn better from the great success of adversarial attacks on images [13–15], other branches of attempts focus on applying point-wise perturbation to change point coordinates, by extending the popular 2D C&W attack [16] and fast gradient sign method (FGSM) [17] attack methods. However, without intentionally considering the geometric properties of point clouds, the extended methods [11, 18] can hardly adapt well, and tend to produce clearly visible outliers, hindering their imperceptibility to humans.
In the view of geometry, point clouds of 3D objects are 2-manifold surfaces embedded in the 3D space [19]. Therefore, attacking point clouds by perturbing their points in the axes can only introduce a small portion of perturbation to change the geometric properties, but leave a large remaining portion to form noise, hindering the imperceptibility. Besides, different regions of point clouds can withstand perturbations of different sizes, e.g., a large modification can be imperceptible at salient regions, but a tiny modification at flat regions is still conspicuous. Therefore, applying perturbations with a uniform magnitude will lead the attacks to be perceptible easily.
To resolve the above issues, we propose a novel NormalAttack framework, which applies curvature-aware shape deformation along normals for imperceptible point cloud attack. First, to directly modify the geometric property of the 2-manifold surfaces, we intentionally enforce the applied perturbation on each point to be concentrated along its normal direction, such that the shape is deformed for attack while allowing a tiny shift along the tangent plane, in which way the deformation is averaged by the local shapes for better imperceptibility. Second, considering the fact that regions with larger curvature can tolerate larger modifications, we devise a curvature-aware attack strategy to guide the perturbation to be concentrated more at these areas, and thus the attack is more imperceptible. We validate the effectiveness of our NormalAttack framework by attacking multiple different deep classification models. Extensive experimental results validate that adversarial point clouds generated by our NormalAttack framework are more imperceptible to those generated by state-of-the-art methods. Besides, we also show NormalAttack is undefendable against adversarial defenses and transferable to unseen classification models, as shown in Figure 1.
Overall, our contribution is at least three-fold.(i)We propose a deformation guiding module that enforces the perturbation to be concentrated along normals to deform the underlying shapes of 3D point clouds for attack.(ii)We propose a curvature-aware module to guide the perturbation to be concentrated more at regions with larger curvature for imperceptibility.(iii)We validate the superiority of the NormalAttack framework to the state-of-the-art methods via extensive experiments on PointNet++, DGCNN, and PointConv.
2. Related Work
2.1. Deep Learning for Point Cloud Classification
Deep learning methods have dominated the mainstream solutions for handling point cloud classification. Early attempts first convert irregular point clouds into structured grid representations, e.g., by projecting point clouds into multiview images [20] or rasterizing into 3D voxel grids [21], and then adopting mature 2D convolutional neural networks. However, these methods either suffer from the loss of detailed geometric information or high computation costs. Therefore, since the pioneering PointNet [22] validated that the structure of multilayer perceptrons (MLPs) followed with maximum pooling can overcome the unorder issue of point clouds, recent solutions focus on learning from point clouds directly. To handle the failure of PointNet in recognizing fine-grained patterns, PointNet++ [2] further captures the fine geometric structure of point clouds by hierarchically applying it to the neighborhood of each point. More solutions include convolution-based KPConv [23], PointCNN [24], and graph-based DGCNN [3]. In this paper, we mainly evaluate the adversarial robustness of several representative point cloud classification models.
2.2. Adversarial Attacks on Deep Learning Models for Point Cloud
Since Szegedy et al. [6] demonstrated the intriguing property of DNNs that an imperceptible perturbation on images can lead them to make mistakes, extensive studies have been made on attacking 2D image classification models [16, 17, 25, 26].
Adversarial attack has been successfully extended to the field of point cloud classification. Due to their unstructured nature, adversarial attack on point clouds can be achieved by adding or deleting points. Xiang et al. [11] performed adversarial attack by adding a limited number of synthetic points, clusters, and objects to the point cloud and showed that PointNet [22] could be fooled in this way. Wicker and Kwiatkowska [27] proposed to determine the critical points in a random and iterative manner and then generated adversarial examples for attack by deleting the critical points. Inspired by the gradient-guided attack method, Yang et al. [28] found key points by calculating the importance scores associated with the labels obtained from the output of the classifier relative to the gradient of the input and then deleted key points in a similar manner. Instead of deleting the points, Zheng et al. [12] devised a more flexible way that moves the points with high saliency towards the center of the shape, such that these points will not influence the surfaces. Another direction of adversarial attack is to perturb point clouds in a similar way as in the field of images. Liu et al. [18] extended the FGSM [17] by adding a -norm constraint to construct imperceptible adversarial 3D point clouds. Lee et al. [29] added adversarial noise to the latent space of an auto-encoder, keeping the decoded shape similar to the original one. To achieve better imperceptibility, Kim et al. [30] proposed to perturb minimal subset of points, instead of all of them. However, very few work exploited the geometric property of point clouds to improve the imperceptibility of generated adversarial point clouds.
2.3. Geometry-Aware Adversarial Attacks
Geometric property is a critical cue for realizing high attack performance and imperceptibility of the point cloud attack task. Tsai et al. [31] incorporated the perturbation constraint into the C&W framework by introducing a k-nearest neighbor loss to ensure the compactness of the local neighborhoods in the obtained adversarial examples. Wen et al. [32] enforced the consistency of local curvature between the adversarial points and benign ones. Both above studies attempt to apply additional geometric constraints passively to achieve high imperceptibility. However, since these constraints are strict, finding a feasible attack solution while satisfying these geometric constraints is usually very challenging. Considering that point clouds are 2-manifold surfaces embedded in the 3D space, we initiatively guide the perturbation to be concentrated along normals, such that very tiny modification can make the underlying shape deformed, and thus leading to better attack performance and imperceptibility. LG-GAN [33] also exploits the manifold property of point clouds. Differently, they enforce the perturbation to be attached to the manifold, while our NormalAttack attempts to destroy the manifold.
We notice a concurrent work [34] that also moves points along normals. Differently, instead of strictly restrict the moving direction, we allow points to be slightly shifted along the tangent plane, such that more feasible solutions can be searched. Besides, the freedom along tangent plane can make the surfaces after perturbation to be smoother. Last but not least, we adopt a curvature-aware perturbation magnitude to further improve the imperceptibility property.
3. Problem Formulation
3.1. Notations
This work considers the setting in a -category point cloud classification problem. Let be an input point cloud containing a set of unordered points sampled from the surface of a 3D object, where each point contains coordinate positions. Let denote the normal of and denote a classifier, e.g., PointNet++ that predicts the category to which the input point cloud belongs.
3.2. Formulation of Adversarial Attack
Suppose can originally correctly classify the category of point cloud where denotes the ground truth label of , adversarial attack aims to find a human-imperceptible perturbation , such that will make an error prediction on the adversarial point cloud:
Note that the above formulation describes the situation of untargeted attack, while targeted attack can be achieved by additionally designating the expected category to be predicted. If not specifically mentioned, we only consider untargeted attack in this paper.
3.3. Traditional Solution for
By borrowing the experience from adversarial attack in the image field, a widely adopted solution is to apply perturbation in the direction which is guided by the gradient, and with the same magnitudewhere is the cost for on the input , is the direction function, and is the perturbation step size.
3.4. Weaknesses of traditional solution
Traditional solutions suffer from at least two main drawbacks. First, the inherent property represented by point clouds is 2-manifold surfaces embed in the 3D space, which is a small subset of the entire 3D Euclidean space. Therefore, noise-like perturbation in the Euclidean space cannot affect the underlying surfaces easily and thus requires to be large, resulting in messy isolated points. Second, different regions of the shapes can withstand different magnitudes of perturbation. Adopting a uniform for all different points will easily lead regions that have lower tolerances, e.g., flat areas, to be perceived by humans after applying perturbation.
3.5. Our Solution for △
To overcome the above drawbacks, we propose to (1) perturb the point clouds mainly along the normal direction of each point, such that the underlying 2-manifold surfaces can be directly modified for better attack performance; and (2) adopt different perturbation magnitude for different points, such that perturbation on regions that have lower tolerance will be suppressed for better imperceptibility. A formal solution is defined as follows:where denotes the perturbation step size for and denotes the perturbation direction is guided by the gradient and .
4. NormalAttack
In this section, we introduce the NormalAttack framework that implements our solution for described in Section 3. We will first present the two main components: the deformation guiding module and the curvature-aware module, and then describe the whole attack framework. Please refer to Figure 2 for demonstration.
4.1. Deformation Guiding Module
To guide the perturbation to be concentrated along normals such that the underlying surfaces are deformed after applying it, we devise a deformation guiding module (DGM).
Instead of strictly restrict normals as the only available moving directions as in ITA [34], DGM applies a much more soft constraint. Suppose is the adversarial point cloud generated from , denotes the corresponding point of in , and denotes the vector from to , the projected perturbation in the tangent direction can be calculated as follows:
Therefore, by enforcing the value of to be small, the perturbation is concentrated along the normal directions.
4.1.1. Discussion with ITA
Compared with ITA [34], our DGM allows larger freedom along the tangent plane, and thus more feasible solutions can be searched. Besides, adding these offsets in the tangent direction can be considered as applying an additional re-sampling process, thus making the surfaces after perturbation more smooth, as shown in Figure 3.
4.2. Curvature-Aware Module
To facilitate a flexible perturbation scheme that allows different perturbation magnitudes for different points, e.g., perturb more on regions that have larger tolerances, we devise a curvature-aware module (CM).
Specifically, CM first calculates the curvature of each point, e.g., ,where is the -nearest neighbors of .
Then, CM calculates the magnitude weight for each point via applying the sigmoid function:where is a temperature scaling parameter.
Therefore, by multiplying the magnitude weight with the original perturbation step size, more perturbation is applied to the regions with larger curvature.
4.3. The Whole Attack Framework
Given the clean point cloud , our NormalAttack framework first randomly initializes the perturbation to form the adversarial point cloud , and then optimize it iteratively, i.e., .
Specifically, the objective loss function of our NormalAttack framework is defined aswhere is the cross-entropy loss for category classification, is the Chamfer distance (CD) loss, and is the Hausdorff distance (HD) loss. By applying gradient descent following (4) iteratively, the adversarial point clouds can be refined viawhere the i-th element of W is w(i). With the help of DGM and CM, NormalAttack deforms the shape of point clouds along normals in a curvature-aware manner, and thus makes adversarial attack imperceptible. Besides, we validate that NormalAttack is also hard to defend and highly transferable in the following experiments.
5. Experiments
5.1. Implementation
For the attack objective function, i.e., (8), we set the weighting parameters with , , and . For the curvature-aware module, we set and . We implement NormalAttack and reproduce all the models with PyTorch and report the results on a workstation with an Intel Xeon E5-2678 [email protected] Hz and 64 GB of memory using a single RTX 2080Ti GPU.
5.2. Experimental Setup
5.2.1. Dataset
We evaluate the attack method on ModelNet40 [35], a dataset that is widely used for 3D point cloud classification tasks and contains 40 of the most common object classes, consisting of 12,311 CAD models, of which 9843 models are used for training and another 2468 for testing.
5.2.2. Models
We choose three representative 3D point cloud classification models, such as PointNet++ [2], DGCNN [3], and PointConv [23] for evaluating that attack performance of our NormalAttack framework. These models are trained on the training data following their original papers.
5.2.3. Baseline Attack Methods
We compare the NormalAttack framework with five baseline attack methods, e.g., FGSM, I-FGSM [36], 3D-ADV [11], GeoA3 [32], and ITA [34]. Note that ITA indicates the method that implements the directional perturbation module of ITA with the adversarial transformation model for black-box attack ablated. Besides, since it is not open-sourced, we reimplement it by ourselves.
5.2.4. Defense Methods
We adopt three adversarial defense methods: statistical outlier removal (SOR) [37], simple random sampling (SRS) [28], and denoiser and upsampler network (DUP-Net) [37]. For SOR, we set the number of points to be removed to be 128; for SRS, we set the number of points to be sampled to be 100; for DUP-Net, we set the number of points in the k-neighborhood to 2, the variance of the allowed point cloud distribution to 1.1, and the minimum number of input points for upsampling to 1024.
5.2.5. Evaluation Metrics
We evaluate the effectiveness of our novel NormalAttack framework using the attack success rate (ASR), i.e., the rate of adversarial samples that can successfully fool the classifiers. Besides, we evaluate the imperceptibility by measuring the perturbation size between the original point clouds and their corresponding adversarial examples using three commonly metrics: -norm distance, Chamfer distance (CD), and Hausdorff distance (HD). Note that these three imperceptibility metrics are measured on the adversarial point clouds generated by these methods that just achieve the best attack success rates in the parameter tuning process, e.g., enlarging perturbation step size and iteration.
5.3. Performance Comparison
5.3.1. Quantitative Results
To demonstrate the imperceptibility of our NormalAttack, we compare the distance metrics with FGSM, I-FGSM [36], 3D-ADV [11], GeoA3 [32], and ITA [34]. The results reported in Table 1 show that all these methods can achieve 100% attack success rates. In particular, our NormalAttack framework requires the lowest CD and HD distances and a medium -norm to achieve it on all three network models, significantly better than state-of-the-art methods. Therefore, we conclude that our proposed NormalAttack framework is imperceptibility.
5.3.2. Visualization Results
To better demonstrate the advantage of our NormalAttack framework in imperceptibility, we visualize the generated adversarial point clouds by different methods in Figure 4. It can be seen that most adversarial point clouds have highly visible outliers, except those generated by ITA and ours, thus validating the usefulness of applying perturbation along normal.
5.3.3. Evaluation on Efficiency
Since efficiency is also an important factor to perform adversarial attacks, we compare of ours with other iterative-based methods, e.g., I-FGSM, 3D-ADV, GeoA3, and ITA on attacking PointNet++. Specifically, we choose the iterations of 20, 40, 60, and 80, and report the attack success rate of all the methods at these iterations in Figure 5. It can be seen that the attack success rates of I-FGSM, GeoA3, ITA, and ours increase with larger iterations, while that of 3D-ADV drop slightly. For all 80 iterations, our NormalAttack achieves the highest attack success rate, and the value is slightly lower than 100% at the 80 iteration, validating the efficiency of NormalAttack. Besides, we also report the perturbation sizes brought by these attack methods at the same iterations in Figure 6. It can be seen that the CD and -norm distances of ours are small while the HD distance of ours is in a moderate level, validating that our NormalAttack framework achieves high attack success rate and imperceptibility at the same time.
5.3.4. Attack performance against defenses
To evaluate the attack performance of our NormalAttack framework against defenses, we compare it with FGSM, I-FGSM, 3D-ADV, GeoA3, and ITA on PointNet++ with applying three adversarial defense methods, i.e., SRS, SOR, and DUP-Net. The results reported in Table 2 show that the attack success rates of all six attack methods, including ours drop after applying any one of the three defense strategies. In particular, I-FGSM, GeoA^3, ITA, and NormalAttack still obtain more than 60% success rates after applying SRS and SOR, validating their effectiveness in handling traditional geometric defense methods. However, after applying the DNN-based DUP-Net, their attack performance drops significantly. In all cases, NormalAttack maintains the largest attack success rates, validating its superiority. To investigate why the performance of state-of-the-art methods drops but ours not, we visualize the results after applying the defense methods in Figure 7. It can be seen that the original outliers generated by these methods for fooling network models are filtered and thus lead to performance drops. Instead, our NormalAttack framework attacks models without bringing clearly visible outliers and thus is only slightly affected by the defenses.
5.3.5. Transferability
To investigate the transferability performance of our NormalAttack framework, we compare it with state-of-the-art iterative-based attack methods, e.g., I-FGSM, 3D-ADV, GeoA3, and ITA, by feeding adversarial point clouds generated by one network model to others. Specifically, we report the adversarial transferability among PointNet++, DGCNN, and PointConv in Table 3. It can be seen that our NormalAttack framework performs the best when transforming from DGCNN to other models, and ranks in the forefront in the other two situations, validating its transferability.
5.4. Ablation Studies and Other Analysis
5.4.1. Curvature-Aware Module
To demonstrate the importance of the curvature-aware module, we compare the results of the full NormalAttack framework with the other framework whose curvature-aware module is ablated with 80 iterations. The results reported in Table 4 show that the attack success rate of the ablated framework is slightly higher than that of the full one, but also with higher distance metrics, validating that the curvature-aware module is critical for maintaining imperceptibility.
5.4.2. Deformation Guiding Module
To demonstrate the importance of the deformation guiding module, we compare the results of the full NormalAttack framework with the other framework whose deformation guiding module is ablated with 80 iterations. The results reported in Table 4 show that the attack success rate of the ablated framework is slightly higher than that of the full one, and the distance metrics are also much higher, validating that the deformation guiding module is critical for maintaining imperceptibility.
5.4.3. Parameter Analysis on t
We also investigate the effects of the temperature scaling parameter in the curvature adaptation module. Specifically, we apply NormalAttack with different values of for 80 iterations to attack PointNet++, DGCNN, and PointConv. The results reported in Table 5 show that both the attack success rate and perturbation size are better when . Therefore, we set in all the experiments.
6. Conclusion
In this paper, we have proposed a novel NormalAttack framework toward imperceptible adversarial attack on point clouds. The key of the framework is to enforce the perturbation to be concentrated along normals to deform the underlying surface of 3D point clouds and perturb more on regions with larger curvature. Extensive experiments validate the effectiveness of NormalAttack. We hope our work can inspire more research on utilizing geometric properties of point clouds to investigate adversarial robustness.
Data Availability
All datasets that support the findings of this study are available publicly.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Keke Tang, Yawen Shi, and Jianpeng Wu contributed equally to this work.
Acknowledgments
This work was supported in part by the National Key Research and Development Project of China (2020AAA0107704), the National Natural Science Foundation of China (62102105, 62073263, and 61902082), Guangdong Basic and Applied Basic Research Foundation (2020A1515110997, 2022A1515011501, and 2022A1515010138), the Science and Technology Program of Guangzhou (202002030263, 202102010419 and 202201020229), the Open Project Program of the State Key Lab of CAD and CG (Grant no. A2218), Zhejiang University.