DeepAI
Log In Sign Up

Double Bubble, Toil and Trouble: Enhancing Certified Robustness through Transitivity

10/12/2022
by   Andrew C. Cullen, et al.
The University of Melbourne
0

In response to subtle adversarial examples flipping classifications of neural network models, recent research has promoted certified robustness as a solution. There, invariance of predictions to all norm-bounded attacks is achieved through randomised smoothing of network inputs. Today's state-of-the-art certifications make optimal use of the class output scores at the input instance under test: no better radius of certification (under the L_2 norm) is possible given only these score. However, it is an open question as to whether such lower bounds can be improved using local information around the instance under test. In this work, we demonstrate how today's "optimal" certificates can be improved by exploiting both the transitivity of certifications, and the geometry of the input space, giving rise to what we term Geometrically-Informed Certified Robustness. By considering the smallest distance to points on the boundary of a set of certifications this approach improves certifications for more than 80% of Tiny-Imagenet instances, yielding an on average 5 % increase in the associated certification. When incorporating training time processes that enhance the certified radius, our technique shows even more promising results, with a uniform 4 percentage point increase in the achieved certified radius.

READ FULL TEXT VIEW PDF

page 8

page 17

02/08/2020

Curse of Dimensionality on Randomized Smoothing for Certifiable Robustness

Randomized smoothing, using just a simple isotropic Gaussian distributio...
05/20/2022

Getting a-Round Guarantees: Floating-Point Attacks on Certified Robustness

Adversarial examples pose a security risk as they can alter a classifier...
02/09/2020

Input Validation for Neural Networks via Runtime Local Robustness Verification

Local robustness verification can verify that a neural network is robust...
03/25/2019

Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness

Adversarial examples are malicious inputs crafted to cause a model to mi...
09/23/2020

Random points are optimal for the approximation of Sobolev functions

We show that independent and uniformly distributed sampling points are a...
02/16/2021

Globally-Robust Neural Networks

The threat of adversarial examples has motivated work on training certif...
02/10/2020

Random Smoothing Might be Unable to Certify ℓ_∞ Robustness for High-Dimensional Images

We show a hardness result for random smoothing to achieve certified adve...

1 Introduction

Learned models, including neural networks, are well known to be susceptible having the output changed by crafted perturbations to an input, that preserve the inputs semantic properties biggio2013evasion . Neural networks not only misclassify these perturbations—known as adversarial examples—but they also assign high confidence to these incorrect predictions. These behaviours have been observed across a wide range of models and datasets, and appear to be a product of piecewise-linear interactions goodfellow2014explaining .

Crafting these adversarial examples typically involves gradient-based optimisation to construct small perturbations. These attacks have been applied to both black- and white-box models papernot2017practical , and can be used to target class changes, to attack all classes dong2018boosting , or even introduce backdoors into model behaviour carlini2017towards . To mitigate the influence of these attacks, defences have typically been designed to minimise the effect of a specific attack (or attacks). Such defences are known as best response strategies in a Stackelberg security game where the defender leads the attacker. Best response defences inherently favour the attacker, as deployed mitigations can be defeated by identifying undefended attack frameworks. Moreover, the defender typically has to incorporate the defence at training time, and as such cannot response reactively to newly developed attacks.

To circumvent these limitations, certified guarantees of adversarial robustness can be constructed to identify class-constant regions around an input instance, that guarantee that all instances within a norm-bounded distance (typically

) are not adversarial examples. Certifications based on randomised smoothing of classifiers around an input point are in a sense optimal 

cohen2019certified : based only on the prediction class scores at the input point, no better radius is in general possible. Despite this, such certifications fail to use readily available—yet still local—information: the certifiability of points nearby to the input of interest. The key insight of this work is that these neighbourhood points may generate certified radius large enough to completely enclose that of a sample point, improving the radius of certification. This process can be extended to use the intersection of the regions of certification of multiple points, and the nature of the input domain itself to generate larger certifications. This leads to our main contribution—Geometrically-Informed Certified Robustness—that enjoys certifications exceeding those of the hitherto best-case guaranteed approach of Cohen et al. (2019) cohen2019certified .

2 Background and literature review

Bounding mechanisms

Conservative bounds upon the impact of norm-bounded perturbations can be constructed by way of either Interval Bound Propagation (IBP) which propagates interval bounds through the model; or Convex Relaxation, which utilise linear relaxation to construct bounding output polytopes over input bounded perturbations salman2019convex ; mirman2018differentiable ; weng2018towards ; CROWN2018 ; zhang2018efficient ; singh2019abstract ; mohapatra2020towards , in a manner that generally provides tighter bounds than IBP lyu2021towards

. In contrast to Randomised Smoothing, bounding mechanisms employ augmented loss functions during training, which promote tight output bounds

xu2020automatic

at the cost of decreased applicability. Moreover they both exhibit a time and memory complexity that makes them infeasible for complex model architectures or high-dimensional data 

wang2021beta ; chiang2020certified ; levine2020randomized .

Randomised smoothing

Outside of bounding mechanisms, another common framework for developing certifications leverages randomised smoothing lecuyer2019certified , in which noise is applied to input instances to smooth model predictions, subject to a sampling distribution that is tied to the -norm of adversarial perturbations being certified against. In contrast to other robustness mechanisms, this application of the noise is the only architectural change that is required to achieve certification. In the case of -norm bounded attacks, Gaussian sampling of the form

(1)

is employed for all test-time instances. These

samples are then used to estimate the expected output of the predicted class of

by way of the Monte-Carlo estimator

(2)

While this Monte Carlo estimation of output expectations under randomised smoothing is a test-time process, model sensitivity to random perturbations may be decreased by performing adversarial training on such random perturbations. To mitigate the computational expense of large sample sizes during each training update, training typically employs single draws from the noise distribution.

Smoothing-based certifications

Based on randomised smoothing, certified robustness can guarantee classification invariance for additive perturbations up to some -norm , with recent work also considering rotational and/or translational semantic attacks li2021tss ; chu2022tpc . -norm certifications were first demonstrated by way of differential privacy lecuyer2019certified ; dwork2006calibrating , with more recent approaches employing Rényi divergence li2018certified , and parametrising worst-case behaviours cohen2019certified ; salman2019provably . By considering the worst-case -perturbations, Cohen et al. (2019) purports that the largest achievable pointwise certification is

(3)

Hhere are the two largest class expectations (as per Equation (2)), is the noise, and

is the inverse normal CDF, or Gaussian quantile function.

3 Geometrically-informed certified robustness

While the work contained within this paper can be applied generally, for this work we will focus upon certifications of robustness about -norm bounded adversarial perturbations, for which we assume that the difficulty of attacking a model is proportional to the size of the certification, based upon the need to evade both human and machine scrutiny gilmer2018motivating . Thus, constructing larger certifications in such a context is inherently valuable.

This specific space is of interest due to both its viability as a defence model, and the provable guarantee that Cohen et al. produces the largest possible certification for any instance cohen2019certified . Over the remainder of this section we will document how it is possible to improve upon this provably best-case guarantee by exploiting several properties of certified robustness.

(a) Transitivity
(b) Multiple Transitivity
(c) Boundary Treatment
Figure 4: Transitive certification exemplars. The Green, Red, and Black circles represent hyperspheres of radius (by Equation 3) about points . The resulting certifications and are described within Equations 5, 12, and 16. The Black line represents the domain boundary.

3.1 Exploiting transitivity

While it is provably true that Equation (3) is the largest achievable certification for any point , it is possible to exploit the behaviour of points in the neighbourhood of in order to enhance certifiable radius. To achieve this, consider the case of a second point , that exists within the certifiable radius of . As both points must correspond to the same class, it then follows that the union of their regions of certification can be also be considered as a region of certification, leading to Definition 3.1.

Definition 3.1 (Overlap Properties of Certification).

A radius of certification about can be calculated by evaluating Equation 3 at . This certification guarantees that no point can induce a chance in the predicted class. That this shape is a -dimensional hypersphere for input data allows us to introduce the notational shorthand

(4)

to represent the region covered by the hypersphere and its surface. It follows from this definition that that if , which ensures that the class predictions at and match, then the region of certification about can be expressed as .

However typically we are concerned not with the size of the region of classification invariance, but rather the distance to the nearest adversarial example. If it is possible to find some such that its region of certification completely encircles that of the certification at , the following definition demonstrates that the certification radius about can be increased.

Lemma 3.2 (Set Unions Certified Radius).

If and have the same class associated with them and , then the nearest possible adversarial example—and thus, the certifiable radius—exists at a distance from , where

(5)
Proof.

The closest point on the surface of to

must exist on the vector between

and . Thus which takes the form of Equation (5). ∎

As such, we can recast the task of constructing a certification from being a strictly analytic function to the nonlinear optimisation problem in terms of a second ball with certified radius centred at

(6)

with Figure (a)a providing a two-dimensional exemplar. Crucially, the above formalism does not require obtaining a global optima, as any yields an improved certification at .

3.2 Multiple transitivity

To further enhance our ability to certify, let us consider the set of points and their associated certifications . If the union of is simply-connected, then the certification across this set can be expressed as , where is the boundary of . This can be further simplified by imposing that and that to ensure that hyperspheres exist near the boundary of and yielding a certification of

(7)

Here is a -dimensional manifold embedded in .

Lemma 3.3 (Optimal positioning of in the case of ).

Consider the addition of a new hypersphere at some point with associated radius , which has an associated boundary . If it is true that

(8)
(9)

then the largest possible certification by Equation 7 is achieved at

(10)
Proof.

The closest point to upon is located at

(11)

where is defined by Equation 6. Thus any improved radius of certification is only achievable if satisfies and Equation 8. Then by symmetry, is the maximally achievable radius of certification if Equation 9 hold and if is defined by Equation 10. ∎

While finding some satisfying Equations 8 and 10 is trivial, proving Equation 9 would require an exhaustive search of the input space . However, even in the absence of such a search, Equation 10 still provides the framework for a simple search for , which follows Figure (b)b.

Lemma 3.4 (Certification from two eccentric hyperspheres).

If is defined by Equation 10 in a fashion that satisfies Equation 8 then an updated certification can be achieved in terms of some defined by Equation 10 by way of

(12)

If Equation 9 holds, then this is the largest achievable certification for .

Proof.

By symmetry we can define the arbitrary rotational mapping from by way of , subject to the condition

(13)

then the intersection of the hyperspheres centred about and occurs at

(14)

This is a consequence of our mapping preserving distances under rotation, giving that , and with the equivalent also holding for .

As a consequence of our choice of coordinate system, it follows that and

(15)

which is an equivalence to Equation (12). ∎

While Equation 7 holds for any , the certification radius beyond cannot be enhanced by adding any one single additional hypersphere without contradicting Lemma 3.4. This is a result of being a -dimensional manifold in , the entirety of which must be enclosed to improve the certification. An example of this can be seen with the two equidistant intersections between (in Red) and (in Black) in Figure (b)b. While multiple spheres could be constructed to completely enclose , the number required grows exponentially with due to the sphere packing kissing number problem coxeter1963upper . This growth in complexity makes adding additional spheres beyond infeasible. Further details of this are contained within Appendix A.2.

3.3 Boundary treatments

Without loss of performance or accuracy, we can freely scale the inputs of neural networks such that . However in the majority of cases a subset of will exist outside . While this observation is trivially true, it has no influence on the radius of certification achieved by Equation 3 due to the symmetry of . However, the asymmetric nature of about guarantees that if exceeds , then the closest point to within the feasible domain must have an associated distance , as is demonstrated within Figure (c)c. This allows us to make the following observation about improving the feasible radius of certification.

Lemma 3.5 (Boundary Certifications by way of Eccentric Circles).

The eccentricity of as a bounding region about , and the potential for a subset of to exist outside the feasible space for instances allows us to construct an updated region of certification where

(16)

where is an indicator function acting upon its operator.

Proof.

In contrast to the prior proof, for this problem we retain the coordinate system of the input space. To support this, we introduce the notation that . If we let , then the intersection between and the bounding surface in dimension creates a bounding hypersphere of the form

(17)

which yields an effective radius .

By denoting the projection of and

onto the bounding hyperplane in the

-th dimension as and , then the distance from to Equation (17) must take the form

(18)

By imposing that when the -th component of is greater than , and otherwise, it follows that must be an improved radius of certification. ∎

3.4 Algorithms

To demonstrate how the above certification approaches can be applied in practice, Algorithm 1 demonstrates the application of Equation (5) through a simple, gradient based solver. Such a solver is highly applicable for solving such problems, due to the inherent smoothing nature of randomised smoothing being applicable both to the function space and its derivatives. To implement the multiple transitivity based approach of Section 3.2, Algorithm 1 can trivially be adapted to evaluate derivatives with respect to Equation (12). The boundary treatment of Section 3.3 does not require any additional calculations, but instead is simply the result of applying Equation (16) to the output of Algorithm 1.

1:  Input: data , samples , iterations , true-label
2:  for  to  do
3:     , , ,
4:     if  then
5:        
6:        if  then
7:           
8:        end if
9:         { calculated by Barzilai-Borwein barzilai1988two . Positive branch is selected if , otherwise the negative branch brings towards the region in which .}
10:     end if
11:  end for
Algorithm 1 Single Bubble Loop.

4 Extracting gradient information from non-differentiable functions

Implementing the aformentioned process requires the ability to evaluate the gradient of the class expectations. This is problematic, as each class expectation is described in terms of a finite sum of non-differentiable indicator functions, as is seen in Equation (2). Within this work we have implemented two mechanisms to circumvent these conditions. The first substitutes the operation with a Gumbel-Softmax jang2016categorical . In doing so, the class expectations are rendered differentiable.

The second approach involves recasting the Monte-Carlo estimators as integrals of the form

(19)

where

is the multivariate-Normal probability distribution centred around

. This formalism, and the symmetry of the underlying space allows for the construction of undifferentiable gradients by

(20)

While this derivation is novel, the resultant gradient operator reflects those seen in prior works salman2019provably . It is important to note that such a sampling process is inherently noisy, and it has previously suggested that the underlying uncertainty scales with the input dimensionality mohapatra2020higher .

The relative performance of these two approaches—respectively labelled ‘Approximate’ and ‘Full’ for the above approach and the Gumbel-Softmax approaches—will be tested in the following section. For the case of the double transitivity of Section 3.2 our experiments suggest that uncertainty in the analytic derivatives produces deleterious results. As such derivatives for the multiple transitivity approach are exclusively considered through autograd for both the Full and Approximate methods.

5 Experiments

Configuration

To evaluate the performance of our proposed certification improvements, we considered the certified radius produced for MNIST

lecun1998gradient

, CIFAR-

krizhevsky2009learning , and Tiny-Imagenet TinyImagenet , the latter of these is a -class variant of Imagenet yang2021Imagenet which downsamples images to . All datasets were modelled using the Resnet

architecture in PyTorch 

NEURIPS2019_9015 , with Tiny-Imagenet also utilising D adaptive average pooling. For both MNIST and CIFAR-, our experimentation utilised a single NVIDIA P GPU core with GB of GPU RAM, with expectations estimated over samples. Training employed Cross Entropy loss with a batch size of over epochs. Each epoch involved every training example was perturbed with a single perturbation drawn from , which was added prior to normalisation. Parameter optimisation was performed with Adam kingma2014adam , with the learning rate set as . Tiny-Imagenet training and evaluation utilised P GPU’s and utilised samples. Training occurred using SGD over epochs, with a starting learning rate of , decreasing by a factor of after and epochs, and momentum set to .

The full code to implement our experiments can be found at https://github.com/andrew-cullen/DoubleBubble.

Certified accuracy

To explore the performance advantage provided by our technique, we begin by considering the performance of Cohen et al. against the best of our approaches using both the approximate and full derivatives, as seen in Figure 5. While there are only minor differences between the two derivative approaches, there are clear regions of out performance relative to Cohen across all tested datasets. The proportion of this increase appears to be tied to the semantic complexity of the underlying dataset, with decreases in predictive accuracy (visible at ) appearing to elicit greater percentage changes in the achieved certified radius, as evidenced by the results for Tiny-Imagenet.

Figure 5: The Certified Accuracy: the proportion of samples correctly predicted and with a certified radius greater than . Blue represents Cohen, while Green and Red respectively represent the best sub-variant utilising either the Approximate or Full derivative approaches. Dashed lines for , solid lines for .

Semantic complexity drives a decrease in the overall confidence of models, inducing a decrease in the separation between the highest class expectations and . While this process shrinks the achievable certified radius, a higher provides more information to inform our gradient based search process, allowing for the identification of larger certifications. This property would suggest that samples with a larger would exhibit a decreased difference between our techniques and that of Cohen et al. would decrease, as smaller values of provide less search information. However, it appears that singularities in the derivatives of as counteract the decreased information provided by the second highest class, leading to the contradictory performance best observed in the MNIST experiments of Figure 5 at .

Rather than strictly considering the best performing of the Approximate and Full solvers, we can also delve into the relative performance of the underlying solvers. Notably there is a moderate increase in the average percentage improvement of Figure 6 between the and cases. This would appear to belie our previous statement regarding larger certifications yielding smaller improvements, due to the asymmetry of class information. However, an equivalent certification for has significantly more information about the second class (due to the multiplicative influence of , allowing for greater improvements from our gradient based search. That there is a clear demarcation between the Full and Approximate solver variants reflects the uncertainties introduced by Equation (20). That the approximate technique is still applicable verifies the utility of our approach even when the final layer is a strict function, rather than a Gumbel-Softmax.

The trends in performance across are further explored in Figures 7 and 9, the latter of which demonstrates that the median performance improvement increases quasi-linearly with . This is driven by both an increase in the performance of the certifications themselves, and in the number of instances able to be certified in an improved fashion. This later property stems from the smoothing influence of , with larger levels of added noise inducing decreases in the difference between the highest class expectations, improving the ability for our search based mechanisms to identify performance improvements. Here increases in the performance of the boundary treatment are correlated with larger radii of certification, due to the multiplicative influence of upon Equation (3).

Figure 6: Percentage improvement in the Certified Radius of Tiny-Imagenet instances relative to Cohen et al.. for varying . This measure presents the median improvement over . Equivalent figures for MNIST and CIFAR- are found in Appendix A.3
Figure 7: Proportion of correctly predicted instances for which each approach yields the highest certification for Tiny-Imagenet. Red, Green, and Orange represent the boundary treatment, Double transitivity, and Single transitivity, with both ties Cohen et al. in Blue. MNIST and CIFAR- results can be seen in Appendix A.3

Numerical performance

The numerical optimisation process at the core of our certification process inherently induces increases in the computational cost of finding these improved certifications, as shown in Table 1. While analytically approximating the derivatives for the first eccentric hypersphere yields a lower certified accuracy, the fact that the corresponding computational cost decreases by a factor of more than emphasises the value of this approach. Interestingly, while the Approximate method does also utilise auto-differentiation for the Double variant, the increase in computational cost from the Single to Double variants is significantly higher than for the Actual approach. This is surprising, as the Approx variant derives smaller values of , which should in turn lead to a smaller, easier to navigate search space for . Instead it counter-intuitively appears that a smaller induces a search space for which is less convex, and more difficult to converge upon.

Dataset Approx. Full
Cohen Single Boundary Double Single Boundary Double
M
C-
T-I
Table 1: Average wall clock time (in seconds) for each computational technique, for a single sample evaluated over draws under noise.
Figure 8: Certified Accuracy comparing the Cohen certification (when trained incorporating MACER) for CIFAR-, as well as the best sub-variant utilising either the Approximate or Full derivative approaches (also employing MACER). Dashed lines for , solid lines for .

Alternative training routines

Recent work has considered the potential for enhancing certified robustness by modifying the underlying training regime to incentivise maximising the expectation gap between classes salman2019provably . One such approach is MACER zhai2020macer , which augments the training time behaviour by considering not just the classification loss, but also the -robustness loss, which reflects proportion of training samples with robustness above a threshold level. Such a training time modification can increase the average certified radius by , however doing so does increase the overall training cost by more than an order of magnitude.

When applying Geometrically-Informed Certified Robustness to models trained with MACER, Figure 8 demonstrates that our modifications yield an even more significant improvement than those observed in Figure 5. Under training with MACER, the best performing of our techniques yielded an approximately percentage point increase in the average certification. From this it is clear that while MACER does enhance the certified radii at a sample point, it also induces enough smoothing in the neighbourhood of the sample point to allow transitivity to yield even more significant improvements than are present without MACER.

However, we must emphasise that while such training time modifications do producer consistently larger certifications, doing so requires significantly more computational resources, both in terms of training time and GPU memory, as compared to the more traditional certification training regime. We also emphasise that training with MACER requires a degree of re-engineering. In contrast the training mechanism used for the remainder of this work only requires the addition of noise to samples prior to being passed through the model, and thus imposes significantly fewer engineering considerations.

Figure 9: Median percentage improvement in the Certified Radius achieved by each of our approaches relative to Cohen et al. for Tiny-Imagenet across the level of additive noise . The median was chosen to provide a fair and representative comparison to Cohen et al.

, that filters out outliers in the percentage improvement when

.

Limitations

While the principles of our Geometrically Informed Certified Robustness are extensible to spaces, our experimental work has so far only considered -norm bounded perturbations due to the guarantee of best possible certification in this space provided by Cohen et al.. Further experimentation could also consider both these general spaces and a broader range of training methods, which have been shown to be able to tighten the achievable radius of certification li2022double .

We note that enhanced robustness certification have the potential to counter beneficial applications of adversarial attacks, such as those used to promote stylometric privacy brennan2012adversarial . However, we believe this drawbacks is significantly outweighed by the potential for enhanced confidence about models for which adversarial incentives exist.

Finally, we also emphasise that our approach requires evaluations of the certified robustness, which each require Monte-Carlo draws, resulting in time- and memory-complexity of and respectively, where

is the size of the output logit vector. With respect to the memory-complexity, this is shared by any randomised smoothing based approach, and could be improved by implementing batching across the Monte-Carlo process. While the time cost can be problematic in some contexts, we emphasise that this framework is both requires both fewer adaptations to the training loop and significantly less training time relative to bound propagation approaches 

levine2020randomized ; shi2021fast . We believe these costs can be reduced by performing the optimisation stages with model draws, and by potentially reusing model draws across the iterative process to approach the time-complexity of prior randomised smoothing based certifications. Even at present, we believe that the increased computational cost is not intractable, especially for human-in-the-loop certifications.

6 Acknowledgements

This research was undertaken using the LIEF HPC-GPGPU Facility hosted at the University of Melbourne. This Facility was established with the assistance of LIEF Grant LE170100200. This work was also supported in part by the Australian Department of Defence Next Generation Technologies Fund, as part of the CSIRO/Data61 CRP AMLC project. Sarah Erfani is in part supported by Australian Research Council (ARC) Discovery Early Career Researcher Award (DECRA) DE220100680.

7 Conclusions

This work has presented mechanisms that can be exploited to improve achievable levels of certified robustness, based upon exploiting underlying geometric properties of robust neural networks. In doing so our Geometrically Informed Certified Robustness approach has been able to generate certifications that exceed prior guarantees by on average more than at , with the percentage increase improving quasi-linearly with . Incorporating training time modifications likes MACER yields more promising results, with the best performing of our approaches yielding a

percentage point increase in the certified proportion at a given radius. Being able to improve upon the size of these guarantees inherently increases the cost of constructing adversarial attacks against systems leveraging machine learning, especially in the case where the attacker has no ability to observe the size of the robustness certificate.

References

  • [1] Jonathan Barzilai and Jonathan M Borwein. Two-Point Step Size Gradient Methods. IMA Journal of Numerical Analysis, 8(1):141–148, 1988.
  • [2] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion Attacks Against Machine Learning at Test Time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 387–402. Springer, 2013.
  • [3] Michael Brennan, Sadia Afroz, and Rachel Greenstadt. Adversarial Stylometry: Circumventing Authorship Recognition to Preserve Privacy and Anonymity. ACM Transactions on Information and System Security (TISSEC), 15(3):1–22, 2012.
  • [4] Fabrício Caluza Machado and Fernando Mário de Oliveira Filho. Improving the Semidefinite Programming Bound for the Kissing Number by exploiting Polynomial Symmetry. Experimental Mathematics, 27(3):362–369, 2018.
  • [5] Nicholas Carlini and David Wagner. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39–57. IEEE, 2017.
  • [6] Ping-yeh Chiang, Renkun Ni, Ahmed Abdelkader, Chen Zhu, Christoph Studer, and Tom Goldstein. Certified Defenses for Adversarial Patches. arXiv preprint arXiv:2003.06693, 2020.
  • [7] Wenda Chu, Linyi Li, and Bo Li. TPC: Transformation-Specific Smoothing for Point Cloud Models. arXiv preprint arXiv:2201.12733, 2022.
  • [8] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified Adversarial Robustness via Randomized Smoothing. In International Conference on Machine Learning, pages 1310–1320. PMLR, 2019.
  • [9] HSM Coxeter. An Upper Bound for the Number of Equal Non-Overlapping Spheres that can Touch Another. In Convexity: Proceedings of the Seventh Symposium in Pure Mathematics of the American Mathematical Society, volume 7, page 53. American Mathematical Soc., 1963.
  • [10] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting Adversarial Attacks with Momentum. In

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    , pages 9185–9193, 2018.
  • [11] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Theory of Cryptography Conference, pages 265–284. Springer, 2006.
  • [12] Justin Gilmer, Ryan P Adams, Ian Goodfellow, David Andersen, and George E Dahl. Motivating the Rules Of The Game for Adversarial Example Research. arXiv preprint arXiv:1807.06732, 2018.
  • [13] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. arXiv preprint arXiv:1412.6572, 2014.
  • [14] Leo A Goodman.

    On Simultaneous Confidence Intervals for Multinomial Proportions.

    Technometrics, 7(2):247–254, 1965.
  • [15] Eric Jang, Shixiang Gu, and Ben Poole. Categorical Reparameterization with Gumbel-Softmax. arXiv preprint arXiv:1611.01144, 2016.
  • [16] Justin Johnson, Fei-Fei Li, and Andrej Karpathy. Tiny ImageNet Visual Recognition Challenge, Accessed 2022-01-10.
  • [17] Diederik P Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980, 2014.
  • [18] Alex Krizhevsky, Geoffrey Hinton, et al. Learning Multiple Layers of Features from Tiny Images. Technical report, University of Toronto, 2009.
  • [19] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  • [20] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Certified Robustness to Adversarial Examples with Differential Privacy. In 2019 IEEE Symposium on Security and Privacy (SP), pages 656–672. IEEE, 2019.
  • [21] Alexander Levine and Soheil Feizi. (de)Randomized Smoothing for Certifiable Defense against Patch Attacks. Advances in Neural Information Processing Systems, 33:6465–6475, 2020.
  • [22] Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. Certified Adversarial Robustness with Additive Noise. In Advances in Neural Information Processing Systems, pages 9459–9469, 2019.
  • [23] Linyi Li, Maurice Weber, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, and Bo Li. TSS: Transformation-Specific Smoothing for Robustness Certification. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 535–557, 2021.
  • [24] Linyi Li, Jiawei Zhang, Tao Xie, and Bo Li. Double Sampling Randomized Smoothing. arXiv preprint arXiv:2206.07912, 2022.
  • [25] Zhaoyang Lyu, Minghao Guo, Tong Wu, Guodong Xu, Kehuan Zhang, and Dahua Lin. Towards Evaluating and Training Rerifiably Robust Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4308–4317, 2021.
  • [26] Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable Abstract Interpretation for Provably Robust Neural Networks. In International Conference on Machine Learning, pages 3578–3586. PMLR, 2018.
  • [27] Jeet Mohapatra, Ching-Yun Ko, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel. Higher-Order Certification for Randomized Smoothing. Advances in Neural Information Processing Systems, 33:4501–4511, 2020.
  • [28] Jeet Mohapatra, Tsui-Wei Weng, Pin-Yu Chen, Sijia Liu, and Luca Daniel. Towards Verifying Robustness of Neural Networks against a family of Semantic Perturbations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 244–252, 2020.
  • [29] Oleg R Musin. The Kissing Number in Four Dimensions. Annals of Mathematics, pages 1–32, 2008.
  • [30] Andrew M Odlyzko and Neil JA Sloane. New Bounds on the number of Unit Spheres that Can Touch a Unit Sphere in n Dimensions. Journal of Combinatorial Theory, Series A, 26(2):210–214, 1979.
  • [31] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical Black-Box Attacks against Machine Learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pages 506–519, 2017.
  • [32] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.

    Pytorch: An Imperative Style, High-Performance Deep Learning Library.

    In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  • [33] Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers. In Advances in Neural Information Processing Systems, 2019.
  • [34] Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks. In Advances in Neural Information Processing Systems, 2019.
  • [35] Kurt Schütte and Bartel Leendert van der Waerden. Das Problem der Dreizehn Kugeln. Mathematische Annalen, 125(1):325–334, 1952.
  • [36] Zhouxing Shi, Yihan Wang, Huan Zhang, Jinfeng Yi, and Cho-Jui Hsieh. Fast Certified Robust Training with Short Warmup. Advances in Neural Information Processing Systems, 34:18335–18349, 2021.
  • [37] Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An Abstract Domain for Certifying Neural Networks. Proceedings of the ACM on Programming Languages, 3(POPL):1–30, 2019.
  • [38] Cristina P Sison and Joseph Glaz. Simultaneous Confidence Intervals and Sample Size Determination for Multinomial Proportions. Journal of the American Statistical Association, 90(429):366–369, 1995.
  • [39] David J Smith and Mavina K Vamanamurthy. How Small is a Unit Ball? Mathematics Magazine, 62(2):101–107, 1989.
  • [40] Shiqi Wang, Huan Zhang, Kaidi Xu, Xue Lin, Suman Jana, Cho-Jui Hsieh, and J Zico Kolter.

    Beta-CROWN: Efficient Bound Propagation with Per-Neuron Split Constraints for Neural Network Robustness Verification.

    Advances in Neural Information Processing Systems, 34, 2021.
  • [41] Lily Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane Boning, and Inderjit Dhillon.

    Towards Fast Computation of Certified Robustness for ReLU Networks.

    In International Conference on Machine Learning, pages 5276–5285. PMLR, 2018.
  • [42] Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond. Advances in Neural Information Processing Systems, 33, 2020.
  • [43] Kaiyu Yang, Jacqueline Yau, Li Fei-Fei, Jia Deng, and Olga Russakovsky. A Study of Face Obfuscation in Imagenet. arXiv preprint arXiv:2103.06191, 2021.
  • [44] Runtian Zhai, Chen Dan, Di He, Huan Zhang, Boqing Gong, Pradeep Ravikumar, Cho-Jui Hsieh, and Liwei Wang. MACER: Attack-free and scalable robust training via maximizing certified radius. In International Conference on Learning Representations, 2020.
  • [45] Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel.

    Efficient Neural Network Robustness Certification with General Activation Functions.

    In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 4939–4948. Curran Associates, Inc., 2018.
  • [46] Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient Neural Network Robustness Certification with General Activation Functions. In Neural Information Processing Systems (NeurIPS), 2018.

Appendix A Appendix

a.1 Algorithmic details

Algorithm 2 supports Algorithm 1 by demonstrating how the class prediction and expectations are calculated. Of note are two minor changes from prior implementations of this certification regime. The first is the addition of the Gumbel-Softmax on line , although this step is only required for the ‘Full’ derivative approach. In contrast th ‘Approximate’ techniques able to circumvent this limitation and can be applied directly to the case where the class election is determined by an .

The second difference to prior works is the calculation of the lower and upper bounds on and on line . Our initial testing revealed that when we employed either Sison-Glaz [38] or Goodman et al. [14] to estimate the multivariate class uncertainties, some Tiny-Imagenet samples devoted more than of their computational time of the process to evaluating the confidence intervals, significantly outweighing even the costly process of model sampling. Further investigation revealed that this was occurring when there were a significant number of classes reporting counts of approximately , the likelihood for which was higher in Tiny-Imagenet due to the increased class count relative to MNIST and CIFAR-. To resolve this, we coalesced all classes where into one single meta-class with an associated class-count , which conforms with the requirements of Goodman et al. [14] that all class counts must be greater than . Our testing demonstrated that while this process slightly decreased the resulting radius of certification (due to small changes in and ), the associated decrease in computational time was significant enough to justify this modification.

1:  Input: Perturbed data , samples , level of added noise
2:  
3:  for i = 1:N do
4:      if {Here is the Gumbel-Softmax}
5:  end for
6:  
7:   { is used as it is differentiable, }
8:   {Calculated by way of Goodman et al. [14]}
9:  
10:  return
Algorithm 2 Class prediction and certification, as required for Algorithm 1

We also note that all the experiments contained within this work have been conducted against publically releaed datasets with established licenses. MNIST exists under a GNU v license; CIFAR- employs a MIT license; and Imagenet employs a BSD Clause license.

a.2 Ramifications of the dimensionality for

To improve the achieved certification in the case , the added set of hyperspheres must fully enclose the -dimensional manifold that marks the intersection between and . In two-dimensions—as is used in the examplar Figure (a)a—this intersection takes the form of two points. If Lemma’s 3.3 and 3.4 are to hold, then encompasing will require two additional certification hyperspheres to be identified.

In the case where , the intersection between these two hyperspheres is the boundary of the circle (equivalent to a hypersphere) with radius

(21)

Thus any set of spheres must uniformly cover all points on the boundary of this surface if we seek to improve the achieved certification.

To provide an indicative example of how the complexity of the region that must be encircled grows with the underlying dimensionality, we now consider some properties of hyperspheres. In higher dimensions, prior work [39] has demonstrated that the volume contained within a -dimensional hypersphere can be expressed as

(22)

with an associated surface area of

(23)

Thus if and are -dimensional hyperspheres, then their region of intersection would in turn be a -dimensional hypersphere, the exterior boundary of which scales with . While may be less than , it should also be true that any additional spheres would likely have an associated radii less than . As such there would appear to be a power-law proportionality with respect to between the area covered by the intersection manifold and the size of spheres over which we would seek to enclose said manifold. This underscores the complexity of finding a set of hyperspheres to encircle the boundary of .

To give further evidence in the growth of complexity, let us consider a unit-hypersphere in that represents the intersection of two hyperspheres in . The task of covering such a hypersphere is similar to that of the sphere packing kissing number  [9], which describes the number of touching-but-not-overlapping unit-hyperspheres that can exist upon the surface of a -dimensional hypersphere. To date, the kissing number has only been solved for the following dimensions outlined in Table 2, however it has been shown to exhibit exponential growth [4].

 [35]  [29]  [30]  [30]
Table 2: Known Kissing numbers for -dimensional hyperspheres

Within the context of this work, the kissing number must be considered to be a significant under-estimate of the number of boundary spheres that would be required to be found, as we must cover all the space around the central sphere (rather than just maximising the number of hyperspheres without intersection), and it is unlikely that the smallest of the set of encircling hyperspheres has the same radius as the region to be encircled. As such, we can be highly confident that the growth in complexity of the task of enclosing the boundary of intersection beyond the set of hyperspheres is exponential.

We must also note that even if it were possible to perform such a bounding operation, the gains in certified radius would be exceedingly minor. If the region of intersection between and was a hypersphere of radius , then going from the case where to would only increase the certified radius from to , which is trivial relative to the increase in computational complexity.

a.3 Relative performance for MNIST and CIFAR-10

While Figure 5 presents the best performing certified accuracy, it is important to understand the relative performance of the Single Transitivity, Double Transitivity, and Boundary treatments, in a similar fashion to Figures 6 and 7. In the case of MNIST, while the percentage increases exhibited in Figure 10 as are broadly similar to their Tiny-Imagenet counterpart for the Approximate solver, the difference between those results and the Full derivative treatment is significantly smaller, especially at . This may, in part, be driven by the samples employed when using MNIST and CIFAR-, in contrast to for Tiny-Imagenet, which should decrease the uncertainty of the gradient estimation steps.

However, the fact that this decreased difference in performance holds for CIFAR- at but not suggests that the performance difference between the techniques is also dependent upon the semantic complexity of the prediction task. While CIFAR- is a more complex predictive environment than MNIST, which should increase the complexity of the gradient based search routine employed within this work, the increased level of noise at has a smoothing influence that decreases the complexity of the search task, and it would appear that this is the primary driver of the relative under performance of the Approximate derivatives in both Tiny-Imagenet and CIFAR- when .

When considering the median percentage improvement (relative to Cohen et al.) of these techniques, MNIST again reveals interesting properties when we consider Figure 14. When compared to CIFAR- and Tiny-Imagenet (in Figures 15 and 9) it becomes apparent that the Approximate approach only produces consistently larger certifications in MNIST. Given the increased uncertainty in the derivatives calculated by the Approximate technique, this would suggest that the approximate solver may be improved by considering common improvements to gradient descent methods like momentum or the addition of calibrated noise.

While MNIST may be the simplest of all the prediction tasks, Figure 12 demonstrates that at low the majority of samples cannot be improved upon by any of the certification enhancements developed within this paper. Given that this does not hold for CIFAR- nor Tiny-Imagenet (in Figure 13 and 7 respectively) this would suggest that the potential for Cohen et al. to be improved upon in low semantic complexity datasets is smaller. That this behaviour is predominantly seen for small also suggests that our initial step size may be too large in these particular cases.

Figure 10: Percentage improvement in the Certified Radius of Tiny-Imagenet instances relative to Cohen et al.. for varying . This measure presents the median improvement over . Equivalent figures for Tiny-Imagenet can be seen in Figure 6
Figure 11: Percentage improvement in the Certified Radius of CIFAR- instances relative to Cohen et al.. for varying . This measure presents the median improvement over . Equivalent figures for Tiny-Imagenet can be seen in Figure 6
Figure 12: Proportion of correctly predicted instances for which each approach yields the highest certification across for MNIST. Red represents the proportion for which the boundary treatment produces the largest certification, with Green, Orange, and Blue representing the same for Double transitivity, Single transitivity, or Cohen et al.. While our approaches subsume Cohen, if no other technique is able to improve upon the base certification, we assign the largest certification as having been calculated by Cohen et al..
Figure 13: Proportion of correctly predicted instances for which each approach yields the highest certification across for CIFAR-. Red represents the proportion for which the boundary treatment produces the largest certification, with Green, Orange, and Blue representing the same for Double transitivity, Single transitivity, or Cohen et al.. While our approaches subsume Cohen, if no other technique is able to improve upon the base certification, we assign the largest certification as having been calculated by Cohen et al..
Figure 14: Median percentage improvement of the Certified Robustness achieved by each of our approaches relative to Cohen et al. for MNIST across the level of additive noise . The median was chosen to provide a fair and representative comparison to Cohen et al., that filters out outliers in the percentage improvement when .
Figure 15: Median percentage improvement of the Certified Robustness achieved by each of our approaches relative to Cohen et al. for CIFAR- across the level of additive noise . The median was chosen to provide a fair and representative comparison to Cohen et al., that filters out outliers in the percentage improvement when .

a.4 Influence of the starting step size

The one heretofore un-considered feature is the influence of the initial step-size within Algorithm 1. As is shown in Figures 16 and 17, while the Full solver only exhibits sensitivity to when , the approximate solvers are far more sensitive, with deleterious performance being observed for in CIFAR-, and even earlier for Tiny-Imagenet. This is likely due to the added uncertainty in the Approximate derivatives leading to convergence upon local sub-optima in the more semantically complex datasets. Based upon these results, the starting step size was uniformly set to for all experiments.

Figure 16: Average delta to the Cohen et al. Certified Radius () for CIFAR- as a function of the initial step size . Here Blue, Green, and Yellow represent the Full Single transitivity, Double transitivity, and Boundary treatments; with Blue, Purple, and Brown representing the same for the Approximate solver.
Figure 17: Average delta to the Cohen et al. Certified Radius () for Tiny-Imagenet as a function of the initial step size . Here Blue, Green, and Yellow represent the Full Single transitivity, Double transitivity, and Boundary treatments; with Blue, Purple, and Brown representing the same for the Approximate solver.