1 Introduction
The advent of machine learning techniques, most notably deep learning
[4], has made complex tasks such as object detection, natural language processing, machine translation, and stockmarket analysis more efficient through their automation. Despite its tremendous success in many academic endeavors and commercial applications, the adoption of deep learning in the perception, decision and control loops of mission/safetycritical systems has been limited. One possible reason for the slow rate of adoption is the limited theoretical understanding of the inner workings of deep neural networks (DNNs). Another possible reason is the lack of guarantees of certain behavioral, and, in particular, robustness properties. One such robustness property is the ability of the network to be resistant to adversarial attacks. In an adversarial attack, the input data is perturbed minimally such that, while the resulting adversarial example closely resembles the unmolested sample, the output of the trained network is affected. Recent work in the deep learning literature
[12, 27, 35, 21, 15, 33, 10]has shown that DNNs may be susceptible to adversarial examples. For stateofart DNN image classifiers, researchers have concocted methods to reliably engineer small perturbations that result in successful adversarial examples. In one such example
[12], the generated adversarial example looks, to the naked human eye, indistinguishable from the original sample, but produces a drastically different classifier output. Some input perturbations are synthetic [12], while others target physical, realworld objects [21, 10].In traditional verification and validation (V&V) processes for safetycritical systems, the robustness of the system is assured via an extensive testing procedure which samples the expected variations in the values of the parameters of the operating environment and system inputs until certain criteria of appropriate coverage metrics are reached. This testbased approach is likely to be less effective for systems with deep learning components since the dimensions of their input spaces are typically much larger. Furthermore, the lack of understanding of the topology of the input spaces involved leads to an open theoretical question: how do we define covergence metrics for systems with deep learning components? Nevertheless, we know that it is feasible to provide formal guarantees of robustness properties of small to medimum DNNs [16] as well as to enforce the guarantee of a robustness property through the training process for larger DNNs [20]. With the increase in predictability that comes with the formal guarantees of adversarial robustness, it is conceivable that deep learning systems could be deployed in mission and/or safetycritical systems.
In this paper, we present a new training method that results in networks which are less sensitive to changes in the input, and, consequently, likely to be more robust to adversarial attacks. The method reliably yields networks of increased robustness by introducing a cost term that penalizes output sensitivity to changes in the input.
The sensitivity measure is described briefly below and is further detailed in Sec. 3. Consider a portion of a twodimensional output space illustrated in Fig. 1 (left) where a segment of the decision boundary between classes and lies. The black dot represents the network output for a sample belonging to . Consider a set
containing all possible perturbation vectors expected upon network deployment. These could include, in the case of a visionbased object classification application, synthetic noise, physical perturbations, shadows, glare, etc. Consider the input data
, and the set resulting from perturbing with all possible disturbances from . The DNN maps the perturbed input set into some output reachable set, an overapproximation of which is represented by the box in the figure. We measure the sensitivity of the network as the aggregate volume of all overapproximations of the output reachable set across the training samples for a predetermined .Also shown in Fig. 1 (left) is the region of potential counterexamples (red portion of the box), which contains all the points in the overapproximated output set for which the classification changes from to , i.e., potential outputs of adversarial inputs. We hypothesize that, by reducing the volume of the output reachable set for some given perturbed input set, the possibility of successful adversarial inputs can be ameliorated, as indicated by the smaller orange box in Fig. 1 (right).
The main contributions of this paper can be summarized as follows:

[leftmargin=*, itemsep=0pt]

We introduce two different approaches to estimate the proposed sensitivity measure for feedforward neural networks (see Sec.
3): one based on Reluplex [16], an update on the classical Simplex method that enables support of ReLU constraints, and one based on evaluating the dual cost of a Linear Programming (LP) relaxation of the exact encoding of ReLU network’s input and output relation
[20, 8]. 
We conduct an empirical study on the relationship between the sensitivities of neural networks and the energy values of their corresponding optimization landscape minima, from which we conclude that networks corresponding to lowerlying minima tend to be less sensitive than those corresponding to highervalued minima. We also show that traditional optimization approaches based on Stochastic Gradient Descent (SGD) algorithms and regularization do not necessarily lead to networks with good sensitivity properties (see Sec.
4). 
Based on the above results, we propose a novel loss function which enables effective taskoriented learning while penalizing high network sensitivity to changes in the input (see Sec. 5). We verify the effectiveness of the proposed cost function on both the Iris and MNIST datasets, and compare the performance of our method with that of stateofart approaches (see Sec. 6).
2 Background and Related Work
Many supervised machine and deep learning algorithms operate based on the principle of Empirical Risk Minimization (ERM), whereby the expectation of a loss function associated with a given hypothesis is approximated with its empirical estimate. The main implication of this modus operandi is the socalled Generalization Error (GE), which refers to the difference between the empirical error and the expected error; practically speaking, the GE manifests itself in a difference in algorithm performance on unseen data relative to the performance observed on the training data. Since ERM is an optimistically biased estimation process [41], techniques to bridge the gap between the empirical risk and the true risk have been proposed: the narrower the gap, the better the generalization properties of the algorithm. Broadly speaking, these techniques fall under one of the following categories:
2.1 Sensitivity and Robustness Analysis and Optimization
The authors of [41] define robustness as the property of a machine learning algorithm to produce similar test and training errors when the test and training samples are similar to each other, and hypothesize that a weak measure of robustness is both sufficient and necessary for good generalizability. Furthermore, they derive a generalization bound that holds under certain constraints. The authors of [25] conclude empirically that trained neural networks are more robust to input perturbations in the vicinity of the training manifold and that higher robustness correlates well with good generalization properties. In [32], the relation between the GE and the classification margin of a neural network is studied. The authors conclude that a necessary condition for good generalization is for the Jacobian matrix of the network to have a bounded spectral norm in the neighborhood of the training samples, in line with the results presented in [25, 42].
2.2 Robustness Against Adversarial Attacks
Adversarial examples are data samples constructed by applying perturbations to known training samples along directions that result in the largest change at the network output [13], so that the perturbed input, often indistinguishable from the original input, leads to an erroneous decision by the network. This vulnerability is directly related to the GE and can be ameliorated by training with adversarial examples [13]. The approach introduced in [14] aims at achieving robustness against adversarial attacks, and consists of stacking a denoising auto encoder (DAE) with the network of interest, the role of the DAE being to map back the adversarial example to the training manifold. An alternative architecture based on contractive auto encoders (CAE) is also proposed. The authors of [20] propose a method to train deep networks based on ReLU activation units that are provably robust to adversarial attacks; they achieve this by minimizing the worstcase loss across the a convex overapproximation of the reachable set under normbounded input perturbations.
2.3 Formal Verification of Neural Networks
Often, the GE includes behavior that is not only incorrect, but also unpredictable, which prevents deployment of deep learning algorithms in safetycritical applications. Formal verification refers to techniques aimed at providing mathematical guarantees about the behavior of systems including computer programs [28]. Formal verification are used most often for safetycritical applications in sectors such as aerospace, nuclear and rail. Early approaches to formally verifying neural networks leveraged SMT solvers [29]
, but did not scale well and were practical on very small networks with a single hidden layer and a small number of neurons. Reluplex
[16] extended the simplex algorithm to support the piecewise linear nature of ReLU activation units, and showed improved scalability properties. More recently, authors have the tackled the problem using optimizationbased approaches including: Mixed Integer Programming (MIP) formulations [6, 7], LP relaxation [9], branch and bound [9, 5], and dual formulations [8, 20]. In contrast to most of the other literature, the framework introduced in [8]applies to a general class of activation functions.
2.4 Analysis of Optimization Landscapes
Modern machine learning methods, most notably deep learning techniques, are posed in the form of highly nonconvex optimization tasks with multiple local minima. The simplest way to quantify the quality of landscape minima is to measure their distance from the global minimum: the closer the energy of a local minimum to that of the global minimum, the better the quality of the minimum and its corresponding network. This definition is illposed as in general the global minimum is not known for a typical nonlinear optimization function. One of the dominant explanations as to why deep learning works so well is that there may be no bad minima at all for the loss functions of deep network [2, 31, 24, 11, 34]: under certain unrealistic assumptions, it has been shown that all the local minima are global minima [17]. In more realistic scenarios, cost functions with a nonvanishing regularization term have been shown to have local minima which are not global minima [22, 36]. Another branch of research leverages optimization methods developed for chemical physics, called the potential energy landscape theory [39]. Using such techniques, the authors of [3] showed that the landscape of the loss function for a specific feedforward artificial neural network with one hidden layer trained on the MNIST dataset exhibited a single funnel structure. In a recent paper [23] (see also, [40, 18]), the measure of the goodness of a minimum is refined by adding further metrics in addition to the value of the cost function or the performance. The work concluded that when an overspecified artificial neural network with one hidden layer and regularization was used to learn the exclusive OR (XOR) function, although the classification error was often zero for various minima, the sparsity structure of the network varied with the minima.
3 Measure of Robustness Via Sensitivity
This section provides a detailed description of the sensitivty measure overviewed in Sec. 1, as well as of the methods that we use to estimate sensitivity for feedforward neural networks with ReLU activation nodes under norm bounded perturbations. Throughout the paper, we use ReLU as the activation function. A feedforward neural network is a parameterized function which maps input data in to output vectors in . By applying norm bounded perturbations to an input data vector , we obtain a perturbed input set , where denotes the perturbation bound. Given the perturbed input set , the output reachable set of the network is . As stated earlier, we use an estimate of the volume of as a surrogate metric for network sensitivity. For feedforward networks with ReLU activation nodes, is generally a nonconvex polytope whose exact volume is difficult to compute. Instead, we define the sensitivity of the network as the volume of a box overapproximation of the output reachable set as illustrated in Fig. 2.
The overapproximation can be either tight (green box with solid boundary in Fig. 2) or loose (red box with dashed boundary in Fig. 2) depending on the computational methods used. There is a significant tradeoff between the conservatism of the overapproximation and the time it takes to compute it. We now describe the two methods used in our experiments. The first method, based on the dual formulation, is used to compute a conservative (i.e., loose) overapproximation. This method turned out to be efficient enough in practice to be incorporated in the training process described in Sec. 5. The second method uses Reluplex, a SMT solver specialized for DNNs, to compute a tight overapproximation of the output polytope.
3.1 Computing Sensitivity Using a Dual Formulation
The volume of the tight overapproximation (green box in Fig. 2) can be computed by finding the min and max values along each dimension of the output vector and then computing the product of the lengths of the segments between such min and max values across all dimensions. Finding the min and max of the output dimensions of a layer ReLU network requires solving a set of difficult optimization problems i.e., minimize and maximize , where is the th entry of the output vector , subject to a set of piecewise linear constraints imposed by the ReLU activations. The min of under norm bounded perturbations to input with bound is the solution of the following optimization problem, where
is the identity matrix
.(1)  
subject to  (2)  
(3)  
(4) 
where is the number of layers in the network, and and are the weights and biases of layer , respectively. The constraints in Eq. 2 capture the fact that the input belongs to the perturbed set . The piecewise linear constraints in Eq. 4 denote the relations between the inputs and outputs of the layers with ReLU activations. The max of is just the negative of the min of or the negative of the solution of the optimization problem in Eq. 1 for . The volume of the overapproximation is:
(5) 
It is known within the operational research community that the optimization problem in 1 can be transformed into a mixedinteger linear programming (MILP) problem through the usage of the big trick [30]. Therefore computing the sensitivity of a network can be done by solving a MILP using a stateofart solver such as Gurobi [26] or by using other techniques with exact encodings benchmarked in a recent work in [5]. However, it remains difficult to scale up the “exact” methods to efficiently verify sensitivity of networks with multiple fullyconnected layers and more than hundreds of ReLU nodes.
Due to the associated computational complexity, using methods with exact encodings is not yet practical for our main goal, which is to incorporate the sensitivity measure into the loss function driving the learning. Instead, we adopt the approach taken by recent work in which ReLU constraints are relaxed into a set of linear constraints. The relaxation is illustrated graphically in Fig. 3.
Consider the piecewise linear constraints from Eq. 4; under the relaxation procedure, they are transformed into a set of linear constraints:
(6) 
With the linear constraints in Eq. 6, the optimization task in Eq. 1 is transformed into a primal linear programming (LP) problem. Furthermore, the dual objective, when evaluated at any feasible point, becomes a lowerbound to the primal objective. As noted in [20], there is a feasible point which can guarantee in practice a “sufficiently tight enough” lower bound to the primal objective.
Let be the dual objective function from [20], where is the input data, is the primal objective matrix, is the norm bound of the allowable input perturbations, is the dual variable, and represents the network parameters; the sensitivity function , which measures the sensitivity of a neural network under normbounded perturbations, is given by:
(7) 
Eq. 7
is the volume of a (box) overapproximation of the output reachable set of a feedforward neural network with ReLU activation nodes under
norm bounded perturbations with bound .The sensitivity function can be evaluated within seconds or less, on a GPUenabled laptop with CUDA, for MNISTsized networks with multiple layers, and more than 1500 ReLU nodes. Comparatively, the sensitivity of a similarsized network when evaluated with an “exact method” such as Reluplex can take hours. However, this computational speed advantage comes at the cost of overestimation of the sensitivity measure, as both the linear relaxation in Eq. 6 and the evaluation of the dual objective at a feasible point adds to the conservatism of the (box) overapproximation.
3.2 Computing Sensitivity Using Reluplex
As mentioned in Sec. 3.1, several “exact” techniques benchmarked in [5] can be used to compute the volume of the tight overapproximation. In the experiments of Sec. 4, we used Reluplex as the solver. To compute the volume of the tight (box) overapproximation, we queried Reluplex the following satisfiability problem: for a given data input and perturbation bound , does there exist a solution that satisfies the set of constraints
(8) 
and Eqs. 2, 3 and 4 in which and are respectively the candidate min and max of . If Reluplex returns a negative answer, then and are valid lower and upper bounds to . Reluplex is queried repeatedly with a sequence of min and max candidates and until a numerical tolerance is reached. We implemented a simple bisection algorithm to generate a sequence of min and max candidates and , and a translational toolchain to repeatedly query Reluplex. As described in more detail in Sec. 4, Reluplex was efficient enough for computing the sensitivities of pretrained networks between the scale of the Iris and MNIST datasets. Fig. 4 plots for close to 200 networks (each corresponding to different landscape minima) in which is the sensitivity computed using the dual formulation and is the sensitivity computed by Reluplex on the same network with the same input perturbation bound. The large peak around network #50 shows that the dual of LP technique can occasionally overestimate the sensitivity of a network by a large amount when compared against Reluplex, but the plot also shows that for the majority of the networks the overestimation remains relatively insignificant.
4 Optimization Landscape Minima: Energy Value vs. Sensitivity
In this section, we investigate the quality of the different minima in the optimization landscape of neural networks by leveraging a combination of energy landscape approaches developed by chemical physicists, and the formal verification tool Reluplex developed by the computer scientists.
4.1 Computing the Minima
Here we give a short description of a procedure to find multiple minima. To find multiple minima, we use methods developed in computational chemistry to explore energy landscapes of atomic and molecular clusters [38]
. First, using different quasiNewton methods such as limited memory BroydenFletcherGoldfarbShanno (LBFGS) and basinhopping methods starting from different random initial guesses, we find multiple local minima. Next, an eigenvector following method is used to find multiple saddle points of index 1 again by feeding different random initial guesses and following the search in the restricted directions where the Hessian matrix has exactly one negative eigenvalue. Then, from each saddle point of index 1, we compute two steepest descent paths in two different directions to connect the corresponding pair of minima. While following this process of connecting pairs of minima, whenever any of the two endpoints of this path is not in the existing database of the minima, it gets added to the database. This process is iterated a few times to obtain multiple minima, while retaining only energetically distinct minima (minima which are at different geometrical locations but have the same energy value are discretesymmetrically related with each other and can be obtained via a discrete transformation of weights).
To run these computations, we use a wrapper around Python Energy Landscape Explorer (PELE) [1]
, which performs energy landscape related computations (i.e., all the aforementioned steps) for any given unconstrained multivariate cost function with continuous variables, as well as Theano
[37], which generates the cost function for the given neural network architecture. To circumvent the discontinuities associated with the ReLU activation functions, we take a constant value whenever the Hessian is singular by using Theano’s builtin gradients libraries. For the Iris data, we fix the regularization parameter to 0.001. Since this is a stochastic method, there is no guarantee that all minima will be found. We only sample the minima to obtain a qualitative picture.4.2 Iris Results and Discussion
The network architecture selected to tackle classification on the Iris dataset consists of 3 fullyconnected hidden layers of 25 neurons each. We used the energy landscape methods described in Sec. 4.1 to obtain 763 Iris minima of varying energy values. The computation took a total of 20 hours on a standard laptop with a single CPU and 8gb of memory.
For each of these minima, we assume a vector of perturbation bounds where , where is a vector containing the ranges of values of features of the entire Iris dataset; that is, each input dimension has its individual perturbation bound proportional to the spread of the input data along that dimension. Given a fixed and an input data point , we queried Reluplex iteratively, as described in Sec. 3.2, until convergence to the min and max of each output dimension is achieved up to some numerical tolerance. The sensitivity of the network output is plotted as a function of the energy value of the landscape minima for in Fig. 5. It can be seen that there is a tendency for networks corresponding to lower energy minima to have lower sensitivity and hence possibly exhibit more robustness to normbounded input perturbations. The plot illustrates the behavior of the average sensitivity for 30 samples and for all samples (150) of the Iris dataset. We observe that there is a very small difference between the average sensitivity of 30 samples versus the average sensitivity of all 150 examples.
5 Learning with Sensitivity Minimization
Having shown empirically in the prior section that lowlying landscape minima lead to networks that are less sensitive than networks with higher energy, we now describe a new training method that encourages convergence to networks with reduced sensivitiy. First, we point out that the traditional taskcentric loss function (e.g., crossentropy) plus an regularization term does not necessarilty guarantee convergence to network with reduced sensitivity. Fig. 6 shows a slew of training runs done on the IRIS dataset. The axis is the sensitivity of the network, and
axis is the training epoch. It can be seen that the change in sensivitiy is irregular and without any clear pattern or trend.
The new training process, which aims to reduce sensitivity of its output networks, is illustrated in Fig. 7.
The main idea of the proposed training process is to augment the usual loss function with an additional weighted term in which the multiplier
is a new hyperparameter, and
is the sensitivity function from 7. Let be the usual loss function with regularization incorporated. Given training sets of size , the optimization task is to find the following minimizer:(9) 
To use the sensitivity function during training, some feasible values for the dual variables need to be chosen. The optimality of the values of the dual variables affects how conservative the estimated sensitivity will be. For the experiments in the next section, we pick the feasible point using Algorithm 1 from [20].
6 Experimental Results
In this section, we present experimental results comparing the performance of three different training methods applied on the standard handwritten digits dataset MNIST. The first method is the baseline which employs a standard cross entropy loss function with regularization. The second method is the weighted sensitivity minimization technique presented in this paper (SM). The third method is the training technique proposed in [20], denoted as K&W, which maximizes the adversarial margin. During each run, at each minibatch step, all 3 training methods are applied on the convolutional network. This produces 3 separate training runs with identical minibatches. The convolutional network architecture has one convolutional layer and two fullyconnected layers. The number of parameters of this architecture is around
. To save some computational time, we keep the network architecture relatively small, although we note that the proposed technique can scale up to larger networks since it exhibits none of the combinatorial complexities present in any of the methods used to compute the tight overapproximation. We used a vanilla stochastic gradient descent (SGD) optimizer without any heuristics based on momentum or RMS propagation. The usage of vanilla SGD, although slower compared to ADAM
[19], was both more numerically stable and sufficient for our purpose, which was to evaluate the relative differences in the evolution of the adversarial robustness, sensitivity and performance of training runs using different techniques on the same set of minibatches. The software environment used for the training was pytorch 0.3.1 and Cuda 9.0. The hardware setup consisted of a laptop with Intel i72500 CPU, 16GB of memory and a single Nvidia GTX 970M graphics card. The learning step size chosen was
, the regularization weight was by default and the batch size was . A perturbation bound of was chosen for all experiments.6.1 Tuning the sensitivity multiplier
As expected, there is a tradeoff between the amount of sensitivity minimization and the network’s classification performance. This tradeoff is observed during the tuning of , the multiplier of the sensitivity term. For a given number of training epochs, increasing the value of typically results in a network that is less sensitive and more robust in terms of the adversarial error measure, but that has worse performance in terms of the classification error. In the experiments, we observed that a good choice of is very much dependent on the dataset. For the Iris dataset experiments, a good choice of was while for the MNIST experiments, a good choice was .
6.2 MNIST Results
For the SM runs, we set the multiplier to 1 and geometrically decrease or increase the until the classification performance of the SM run approached that of the baseline. This iterative process settled on and the resulting performance evaluations of 3 training runs are shown in Figs. 8 and 9.
Fig. 8 shows the classification error of all 3 training runs evaluated on the test data. With , the classification error performance of our training method SM is nearly as good as the baseline and exceeds the performance of K&W.
Next, we look at the sensitivity measure of the 3 training runs evaluated on the test data. This is the measure which our training method tries to minimize. As shown in Fig. 9, not surprisingly, SM outperforms both the baseline and K&W. Note that the multiplier term was small enough that the sensitivity actually increases in the run produced by our own training method.
Finally we look at the adversarial error, i.e., the robustness measure from [20], which is the percentage of the test data for which the application of bounded perturbations could lead to adversarial examples. As shown in Fig. 10, each of the 3 training runs are evaluated for their adversarial error on the test data. Here, SM was outperformed by K&W; however, when compared against the baseline, both methods increase the robustness of the network by significant margins.
6.3 Discussion
Figs. 9 and 10 show a possible negative correlation between the sensitivity of the network and its adversarial robustness. While our method does not minimize the adversarial error directly, the empirical outcome from Fig. 10 seems to indicate that encouraging small sensitivity has an effect on that measure. Complementarily, the method from [20] does directly minimize the adversarial error by maximizing of the adversarial margin; however, as shown in Fig. 9 it appears to also have significant effect towards minimizing the sensitivity of the network when compared against the baseline. The difference in classification error between the method in [20] and our method could be due to the former leading to overfitting due to the learning leveraging the worstcase adversarial example; in contrast, our training method reduces the size of the output reachable set more uniformly, and not based on any particular sampling scheme at the output. Alternatively, if the sensitivity term in our training method is replaced by the negative of the adversarial margin of K&W, it is conceivable that we will observe a similar tradeoff between their robustness measure and the classification performance.
7 Conclusion and Future Work
In this paper, we studied the relationship between the energy value of the network minima and the sensitivity of the resulting network. Using formal verification techniques, we have demonstrated on the Iris dataset a direct correlation between the sensitivity of the network and the energy value of the corresponding landscape minima. Based on that result, we also studied the relationship between the sensitivity of the neural network and the adversarial robustness of the network. We have found, through a novel learning framework, that the adversarial robustness of the network can be improved via a reduction of the sensitivity of the network. According to the proposed learning approach, the sensitivity and/or robustness constraints are handled via the penalty method, which results in the usual unconstrained optimization problem solved in machine learning. The resulting penalty parameter is a hyperparmeter which can be tuned to reduce the degradation in the performance of the network while still providing improvements in the adversarial robustness of the network.
At this point, we have only considered perturbations of small magnitude captured by the normbound mathematical abstraction. The extension of the verification technique to complex additive disturbances such as occlusion, glare, and shadow or multiplicative disturbances such as rotation and translation, remains an open question as the mathematical abstractions of those complex disturbances may not fit with the dual of LP relaxation framework. Furthermore, the certification of any new process for guaranteeing robustness of deeplearning based components in the context of mission and safetycritical applications remains an open question as well.
Other questions remain as to the scalability and generalization of the training method. The key benchmark of scalability is whether sensitivity, robustness, and any other desired properties can be efficiently computed for significantly larger networks, using the dual of LP relaxation formulation. The generalization questions refers to whether the training technique produces the same consistent acrossthe board convergence to the desired property enforced via the penalty term for other datasets. Beyond robustness and/or sensitivity, other formalizable properties of the neural network that are critical to building an assurance case need to be studied as well. There is also a need to address the conservatism of the sensitivity computations, and whether the looseness of the approximation can be further improved without significant reduction in the efficiency of the method.
References
 [1] https://pelepython.github.io/pele/.

[2]
P. Baldi and K. Hornik.
Neural networks and principal component analysis: Learning from examples without local minima.
Neural networks, 2(1):53–58, 1989.  [3] A. Ballard, S. Martiniani, D. Mehta, J. Stevenson, and D. J. Wales. Energy landscapes for machine learning. Phys. Chem. Chem. Phys., 19:12585–12603, 2017.
 [4] Y. Bengio, I. J. Goodfellow, and A. Courville. Deep learning. MIT Press., 2015.
 [5] R. Bunel, I. Turkaslan, P. H. Torr, P. Kohli, and M. P. Kumar. Piecewise linear neural network verification: a comparative study. arXiv preprint arXiv:1711.00455, 2017.
 [6] C.H. Cheng, G. Nührenberg, and H. Ruess. Maximum resilience of artificial neural networks. In International Symposium on Automated Technology for Verification and Analysis, pages 251–268. Springer, 2017.
 [7] S. Dutta, S. Jha, S. Sanakaranarayanan, and A. Tiwari. Output range analysis for deep neural networks. arXiv preprint arXiv:1709.09130, 2017.
 [8] K. Dvijotham, R. Stanforth, S. Gowal, T. A. Mann, and P. Kohli. A dual approach to scalable verification of deep networks. CoRR, abs/1803.06567, 2018.
 [9] R. Ehlers. Formal verification of piecewise linear feedforward neural networks. In International Symposium on Automated Technology for Verification and Analysis, pages 269–286. Springer, 2017.
 [10] I. Evtimov, K. Eykholt, E. Fernandes, T. Kohno, B. Li, A. Prakash, A. Rahmati, and D. Song. Robust physicalworld attacks on deep learning models. arXiv preprint arXiv:1707.08945, 1, 2017.
 [11] I. Goodfellow, O. Vinyals, and A. Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544, 2014.
 [12] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples (2014). arXiv preprint arXiv:1412.6572.
 [13] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. CoRR, abs/1412.6572, 2014.
 [14] S. Gu and L. Rigazio. Towards deep neural network architectures robust to adversarial examples. CoRR, abs/1412.5068, 2014.
 [15] S. Huang, N. Papernot, I. Goodfellow, Y. Duan, and P. Abbeel. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.02284, 2017.
 [16] G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pages 97–117. Springer, 2017.
 [17] K. Kawaguchi. Deep learning without poor local minima. arXiv preprint arXiv:1605.07110, 2016.
 [18] K. Kawaguchi, L. P. Kaelbling, and Y. Bengio. Generalization in deep learning. arXiv preprint arXiv:1710.05468, 2017.
 [19] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
 [20] J. Z. Kolter and E. Wong. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint arXiv:1711.00851, 2017.
 [21] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533, 2016.
 [22] D. Mehta, T. Chen, T. Tang, and J. D. Hauenstein. The loss surface of deep linear networks viewed through the algebraic geometry lens. 2018.
 [23] D. Mehta, X. Zhao, E. A. Bernal, and D. J. Wales. Loss surface of xor artificial neural networks. Phys. Rev. E, 97:052307, May 2018.
 [24] Q. Nguyen and M. Hein. The loss surface of deep and wide neural networks. arXiv preprint arXiv:1704.08045, 2017.
 [25] R. Novak, Y. Bahri, D. A. Abolafia, J. Pennington, and J. SohlDickstein. Sensitivity and generalization in neural networks: an empirical study. In International Conference on Learning Representations, 2018.
 [26] G. Optimization. Inc.,gurobi optimizer reference manual, 2015. URL: http://www. gurobi. com, 2014.
 [27] N. Papernot, N. Carlini, I. Goodfellow, R. Feinman, F. Faghri, A. Matyasko, K. Hambardzumyan, Y.L. Juang, A. Kurakin, R. Sheatsley, et al. cleverhans v2. 0.0: an adversarial machine learning library. arXiv preprint arXiv:1610.00768, 2016.
 [28] D. Peled. Software Reliability Methods. Texts in Computer Science. Springer, 2001.
 [29] L. Pulina and A. Tacchella. Challenging smt solvers to verify neural networks. AI Commun., 25:117–135, 2012.
 [30] A. Richards and J. How. Mixedinteger programming for control. In American Control Conference, 2005. Proceedings of the 2005, pages 2676–2683. IEEE, 2005.
 [31] A. Saxe, J. McClelland, and S. Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120, 2013.
 [32] J. Sokolic, R. Giryes, G. Sapiro, and M. R. D. Rodrigues. Robust large margin deep neural networks. IEEE Transactions on Signal Processing, 65:4265–4280, 2017.
 [33] L. Song and P. Mittal. Inaudible voice commands. arXiv preprint arXiv:1708.07238, 2017.
 [34] D. Soudry and Y. Carmon. No bad local minima: Data independent training error guarantees for multilayer neural networks. arXiv preprint arXiv:1605.08361, 2016.
 [35] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
 [36] A. Taghvaei, J. W. Kim, and P. Mehta. How regularization affects the critical points in linear networks. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 2502–2512. Curran Associates, Inc., 2017.
 [37] Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv eprints, abs/1605.02688, May 2016.
 [38] D. J. Wales. Energy Landscapes. Cambridge University Press, Cambridge, 2003.
 [39] D. J. Wales. Some further applications of discrete path sampling to cluster isomerization. Mol. Phys., 102:891–908, 2004.
 [40] L. Wu, Z. Zhu, and W. E. Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes. ArXiv eprints, June 2017.
 [41] H. Xu and S. Mannor. Robustness and generalization. CoRR, abs/1005.2243, 2010.
 [42] Y. Yoshida and T. Miyato. Spectral norm regularization for improving the generalizability of deep learning. CoRR, abs/1705.10941, 2017.
Comments
There are no comments yet.