Verifying Low-dimensional Input Neural Networks via Input Quantization

08/18/2021
by   Kai Jia, et al.
MIT
0

Deep neural networks are an attractive tool for compressing the control policy lookup tables in systems such as the Airborne Collision Avoidance System (ACAS). It is vital to ensure the safety of such neural controllers via verification techniques. The problem of analyzing ACAS Xu networks has motivated many successful neural network verifiers. These verifiers typically analyze the internal computation of neural networks to decide whether a property regarding the input/output holds. The intrinsic complexity of neural network computation renders such verifiers slow to run and vulnerable to floating-point error. This paper revisits the original problem of verifying ACAS Xu networks. The networks take low-dimensional sensory inputs with training data provided by a precomputed lookup table. We propose to prepend an input quantization layer to the network. Quantization allows efficient verification via input state enumeration, whose complexity is bounded by the size of the quantization space. Quantization is equivalent to nearest-neighbor interpolation at run time, which has been shown to provide acceptable accuracy for ACAS in simulation. Moreover, our technique can deliver exact verification results immune to floating-point error if we directly enumerate the network outputs on the target inference implementation or on an accurate simulation of the target implementation.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/06/2020

Exploiting Verified Neural Networks via Floating Point Numerical Error

We show how to construct adversarial examples for neural networks with e...
11/20/2015

Resiliency of Deep Neural Networks under Quantization

The complexity of deep neural network algorithms for hardware implementa...
12/08/2021

Neural Network Quantization for Efficient Inference: A Survey

As neural networks have become more powerful, there has been a rising de...
11/25/2021

QNNVerifier: A Tool for Verifying Neural Networks using SMT-Based Model Checking

QNNVerifier is the first open-source tool for verifying implementations ...
03/26/2019

Robustness of Neural Networks to Parameter Quantization

Quantization, a commonly used technique to reduce the memory footprint o...
03/02/2019

Verifying Aircraft Collision Avoidance Neural Networks Through Linear Approximations of Safe Regions

The next generation of aircraft collision avoidance systems frame the pr...
01/17/2022

Neural Network Compression of ACAS Xu Early Prototype is Unsafe: Closed-Loop Verification through Quantized State Backreachability

ACAS Xu is an air-to-air collision avoidance system designed for unmanne...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The Airborne Collision Avoidance System (ACAS) is crucial for aircraft safety [10]

. This system aims to avoid collision with intruding aircraft via automatically controlling the aircraft or advising a human operator to take action. The ACAS typically takes low-dimensional sensory inputs, including distance, direction, and speed for the intruder and ownship aircraft, and provides a control policy which is a valuation for a set of candidate actions such as “weak left” or “strong right”. Recent work has formulated aircraft dynamics under uncertainties such as advisory response delay as a partially observable Markov decision process for which dynamic programming can be used to compute values for different actions 

[11]. The value function computed via dynamic programming is often stored in a lookup table with millions of entries [12] that require gigabytes of storage. While this table could, in principle, be used to implement the ACAS, the high storage demand makes it too costly to be embedded in practical flight control systems. This situation has motivated the development of table compression techniques, including block compression with reduced floating-point precision [13]

and decision trees

[6].

Recently, neural networks have emerged as an efficient alternative for compressing the lookup tables in ACAS Xu (ACAS X for unmanned aircraft) by approximating the value function with small neural networks. Specifically, Julian et al. [6] compresses the two-gigabyte lookup table into 45 neural networks with 2.4MB of storage, where each network handles a partition of the input space.

Katz et al. [9] proposes a set of safety properties for the ACAS Xu networks, such as that a “strong right” advisory should be given when a nearby intruder is approaching from the left. These safety properties have served as a valuable benchmark to motivate and evaluate multiple verification algorithms [9, 19, 15, 17, 1]. Such verifiers typically need to perform exact or conservative analysis of the internal neural network computation [14, 18]. Unfortunately, neural network verification is an NP-Complete problem [9], and therefore the verifiers need exponential running time in the worst case and can be very slow in practice. In particular, Bak et al. [1] recently presented the first verifier that is able to analyze the properties to in the ACAS Xu benchmarks with a time limit of 10 minutes for each case, but their verifier still needs 1.7 hours to analyze the property .

In summary, previous techniques perform the following steps to obtain and verify their neural network controllers for ACAS:

  1. Compute a lookup table containing the scores of different actions given sensory states via dynamic programming.

  2. Train neural networks to approximate the lookup table.

  3. In deployed systems, use the neural networks to provide control advisories.

    • At run time, the networks give interpolated scores for states not present in the original lookup table.

    • Neural network verifiers that analyze the internal computing of neural networks are adopted to check if the networks meet certain safety specifications.

We propose instead to verify neural networks with low-dimensional inputs, such as the ACAS Xu networks, via input quantization and state enumeration. Specifically, we prepend a quantization layer to the network so that all the internal computation is performed on the discretized input space. Our proposed technique performs the following steps to obtain and verify a quantized neural network:

  1. We take a pretrained network and prepend an input quantization layer to the network. The input quantization should be compatible with the original lookup table, i.e., preserving the grid points in the lookup table.

  2. In deployed systems, sensory inputs are first quantized by the input quantization layer. The original network then computes the scores for the quantized input.

    • At run time, the quantization process is equivalent to nearest-neighbor interpolation.

    • To verify the network for any specification, we enumerate all quantized states within the constraint of the specification and check if the network outputs meet the specification.

Our method provides the following desirable features:

  1. Our method provides acceptable runtime accuracy for ACAS Xu. Our input quantization is equivalent to nearest-neighbor interpolation and gives identical results on the table grid points as the original continuous network. Julian et al. [6] has shown that nearest-neighbor interpolation on the lookup table for runtime sensory inputs provides effective collision avoidance advisories in simulation.

  2. Our method enables efficient verification. Verifying the input-quantized networks for any safety specification takes nearly constant time bounded by evaluating the network on all the grid points in the quantized space. Multiple specifications can be verified simultaneously by evaluating the network on the grid once and checking the input and output conditions for each property. Our method provides a verification speedup of tens of thousands of times compared to the ReluVal [19] verifier.

  3. Many existing verifiers do not accurately model floating-point arithmetic due to efficiency considerations, thus giving potentially incorrect verification results

    [5]. For example, Wang et al. [19] reports that Reluplex [9] occasionally produces false adversarial examples due to floating-point error.

    By contrast, our verification result is exact (i.e., complete and sound) and does not suffer from floating-point error because we combine input quantization and complete enumeration of the effective input space. Moreover, input quantization allows directly verifying on the target implementation or an accurate simulation of the implementation, and therefore provides trustworthy safety guarantees for given neural network inference implementations.

  4. Our technique allows easily verifying more complicated network architectures, such as continuous-depth models [2]. Our verification only needs an efficient inference implementation for the networks. By contrast, extending other neural network verifiers to new network architectures requires significant effort.

We recommend input quantization for neural networks with low-dimensional inputs as long as the quantization provides sufficient accuracy for the target application and the quantization space is small enough to allow efficient enumeration. This technique enables efficient, exact, and robust verification and provides reliable performance on the deployed platform.

2 Method

We formally describe our input-quantization method. This paper uses bold symbols to represent vectors and regular symbols to represent scalars. The superscript represents derived mathematical objects or exponentiation depending on the context.

Let denote the computation of a neural network on -dimensional input space with being a small number. We propose to use a quantized version of the network for both training and inference, defined as

(1)

where is the quantization function such that with being a finite-sized set. For a specification where and are predicates, we verify regarding by checking:

(2)
where

Since , the complexity of verifying is bounded by .

We quantize each dimension of independently via . Note that if some of the dimensions are highly correlated in some application, we can quantize them together to avoid a complete Cartesian product and thus reduce the size of the quantized space.

In many cases, the input space is uniformly quantized. Previous work has utilized uniform input quantization for neural network verification [20, 4] and uniform computation quantization for efficient neural network inference [3]. Given a quantization step and a bias value , we define a uniform quantization function as:

(3)

where denotes rounding to the nearest integer.

The values of are essentially determined according to prior knowledge about the target application and may thus be nonuniform. Let denote the range of . We use nearest neighbor for nonuniform quantization:

(4)

The ACAS Xu networks are trained on a lookup table , where the domain is a finite set. We choose the quantization scheme so that the quantization preserves grid points, formally . In this way, the training processes of and are identical. In fact, we directly prepend as an input quantization layer to a pretrained network to obtain . Note that we can use a denser quantization than the grid points in so that prediction accuracy might get improved by using the neural network as an interpolator.

3 Experiments

We evaluate our method on checking the safety properties for the ACAS Xu networks [9]. Note that accuracy of input-quantized networks in deployed systems is acceptable since the quantization is equivalent to nearest-neighbor interpolation that has been shown to provide effective collision avoidance advisories in simulation [6].

Experiments in this section focus on evaluating the runtime overhead of input quantization and the actual speed of verification by enumerating quantized states. We train two networks of different sizes to evaluate the scalability of the proposed method.

3.1 Experimental Setup

The horizontal CAS problem takes seven inputs as described in Table 1, and generates one of the five possible advisories: COC (clear of conflict), WL (weak left), WR (weak right), SL (strong left), and SR (strong right).

Symbol Description Values in the lookup table
(m) Distance from ownship to intruder 32 values between 0 and 56000 1
(rad) Angle to intruder 2 41 evenly spaced values between and
(rad) Heading angle of intruder 2 41 evenly spaced values between and
(m/s) Speed of ownship
(m/s) Speed of intruder
(sec) Time until loss of vertical separation
Previous advisory {COC, WL, WR, SL, SR}
Table 1:

Description of horizontal CAS inputs. The last column describes the values used to generate the lookup table, which are taken from the open-source implementation of HorizontalCAS

[7] and the Appendix VI of Katz et al. [9].

Julian et al. [8] proposes to train a collection of neural networks where each network works with a pair of specific

values, takes the remaining five values as network inputs, and approximates the corresponding scores in the lookup table. Although ACAS only needs to suggest the action with the maximal score, the network is still trained to approximate the original scores in the table instead of directly giving the best action because the numerical scores are used in a Kalman filter to improve system robustness in the face of state measurement uncertainty 

[8]. In order to maintain the action recommendation of the original table while reducing score approximation error, Julian et al. [8]

adopts an asymmetric loss function that imposes a higher penalty if the network and the lookup table give different action advisories.

Katz et al. [9] proposes a few ACAS Xu safety properties as a sanity check for the networks trained by Julian et al. [8]. These properties have also served as a useful benchmark for many neural network verifiers. Although the pretrained networks of Julian et al. [8] are publicly accessible, the authors told us that they could not provide the training data or the source code due to regulatory reasons. They suggested that we use their open-source HorizontalCAS system [7] to generate the lookup tables to train our own networks. However, HorizontalCAS networks differ from the original ACAS Xu networks in that they only have three inputs by fixing and . We modified the source code of HorizontalCAS to match the input description in Table 1 so that we can directly use the ReluVal [19] verifier.

We evaluate our method by analyzing the property proposed in Katz et al. [9], which usually takes the longest time to verify among all the properties for many verifiers [9, 19, 16]. Other properties share a similar form but have different input constraints and output requirements. Note that property is the most compatible with the open-source HorizontalCAS because the input constraints of other properties are beyond the ranges in Table 1. For example, property has but the quantization scheme of for the original ACAS Xu networks is not publicly available.

The specification of is:

  • Description: Even if the previous advisory was “weak right”, the presence of a nearby intruder will cause the network to output a “strong left” advisory instead.

  • Tested on: the network trained on and

  • Input constraints: , , , , .

We conduct the experiments on a workstation equipped with two GPUs (NVIDIA Titan RTX and NVIDIA GeForce RTX 2070 SUPER), 128 GiB of RAM, and an AMD Ryzen Threadripper 2970WX processor. We train two neural networks for property (i.e., with and

) with PyTorch.

Our small network has five hidden layers with 50 neurons in each layer, and our large network has seven hidden layers with 100 neurons in each layer. We use the ReLU activation.

We implement the nearest-neighbor quantization for via directly indexing a lookup table. The greatest common divisor of differences between adjacent quantized values is 5. Therefore, we precompute a lookup table such that is the nearest neighbor of in the set of quantized values. We use the torch.index_select operator provided by PyTorch to take elements in the lookup table in a batched manner. Other network inputs use uniform quantization as described in Table 1. We implement uniform quantization according to the equation (3).

3.2 Experimental Results

Metric Small network     Large network
Policy accuracy 96.87% 98.54%
Score error 0.052 0.026
Score error
Table 2: Accuracies achieved by the networks evaluated on the lookup table. For comparison, Julian and Kochenderfer [7] reports an accuracy of 97.9% for networks trained only with three out of the five inputs (they fixed and ). This table shows that our network achieves sufficient accuracy for practical use.

Verification method Small network     Large network ReluVal [19] 0.622 171.239 Input quantization - specific 1 0.002 0.002 Input quantization - all 2 0.384 0.866

  • Network is evaluated on the 60 input states that fall within the input constraint of .

  • Network is evaluated on all the 860,672 input states in a batched manner. This time is the upper bound for verifying any first-order specification in the form of by ignoring the time on evaluating predicates and .

Table 3: Comparing verification time (in seconds) for the property on two methods: the ReluVal verifier [19] that runs on multiple cores, and exhaustive enumeration in the quantized input space on a single CPU core. This table shows that verification by enumerating quantized input states is significantly faster in our case and also more scalable regarding different network sizes.

Let (resp. ) denote the scores given by the network (resp. the original lookup table) for the five candidate actions on the

lookup table entry. We consider three accuracy measurements, assuming a uniform distribution of the table index

:

  • Policy accuracy

    is the probability that the network recommends the same action as the original lookup table. Formally, it is

    .

  • Score error measures the error of approximated scores, defined as , where .

  • Score error measures the error of approximated scores, defined as , where .

Table 2 presents the accuracies achieved by our networks, which shows that our training achieves comparable results as the HorizontalCAS system [7].

To verify the networks, we prepend them with an input quantization layer that implements the quantization scheme given in Table 1. To verify any specification or a set of specifications, we evaluate the network on all the 860, 672 points in the quantized space and check if each input/output pair meets the specification(s). Evaluating the network on the grid points takes 0.384 seconds for the small network and 0.866 seconds for the large one. We evaluate the network on multiple inputs in a batched manner to benefit from optimized numerical computing routines included in PyTorch. Adding the quantization layer incurs about 2% runtime overhead. We do not do any performance engineering and use the off-the-shelf implementation provided by PyTorch. Our verification speed can be further improved by using multiple CPU cores or using the GPU.

We also compare our method with ReluVal [19] on verifying the property . The input constraint of consists of only 60 states in the quantized space. Therefore, we only need to check if the network constantly gives the “weak right’ advisory for all the 60 states to verify . As shown in Table 3, input quantization significantly reduces the verification time compared to the ReluVal solver.

4 Conclusion

This paper advocates input quantization for the verification of neural networks with low-dimensional inputs. Our experiments show that this technique is significantly faster and more scalable than verifiers that analyze the internal computations of the neural networks on verifying ACAS Xu networks. Moreover, our method does not suffer from the floating-point discrepancy between the verifier and the network inference implementation. In general, our method applies to deterministic floating-point programs that take low-dimensional inputs as long as the target application tolerates input quantization such that enumerating all the quantized values takes acceptable time.

References

  • [1] S. Bak, H. Tran, K. Hobbs, and T. T. Johnson (2020) Improved geometric path enumeration for verifying relu neural networks. In International Conference on Computer Aided Verification, pp. 66–96. Cited by: §1.
  • [2] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud (2018)

    Neural ordinary differential equations

    .
    In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31, pp. . External Links: Link Cited by: item 4.
  • [3] A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer (2021) A survey of quantization methods for efficient neural network inference. arXiv preprint arXiv:2103.13630. Cited by: §2.
  • [4] K. Jia and M. Rinard (2020)

    Efficient exact verification of binarized neural networks

    .
    In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33, pp. 1782–1795. External Links: Link Cited by: §2.
  • [5] K. Jia and M. Rinard (2020) Exploiting verified neural networks via floating point numerical error. arXiv preprint arXiv:2003.03021. Cited by: item 3.
  • [6] K. D. Julian, M. J. Kochenderfer, and M. P. Owen (2019) Deep neural network compression for aircraft collision avoidance systems. Journal of Guidance, Control, and Dynamics 42 (3), pp. 598–608. Cited by: item 1, §1, §1, §3.
  • [7] K. D. Julian and M. J. Kochenderfer (2019) Guaranteeing safety for neural network-based aircraft collision avoidance systems. In 2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC), pp. 1–10. Cited by: item 1, §3.1, §3.2, Table 1, Table 2.
  • [8] K. D. Julian, J. Lopez, J. S. Brush, M. P. Owen, and M. J. Kochenderfer (2016) Policy compression for aircraft collision avoidance systems. In 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC), Vol. , pp. 1–10. External Links: Document Cited by: §3.1, §3.1.
  • [9] G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer (2017) Reluplex: an efficient smt solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pp. 97–117. Cited by: item 3, §1, §3.1, §3.1, Table 1, §3.
  • [10] M. J. Kochenderfer and J. Chryssanthacopoulos (2011) Robust airborne collision avoidance through dynamic programming. Massachusetts Institute of Technology, Lincoln Laboratory, Project Report ATC-371 130. Cited by: §1.
  • [11] M. J. Kochenderfer, C. Amato, G. Chowdhary, J. P. How, H. J. D. Reynolds, J. R. Thornton, P. A. Torres-Carrasquillo, N. K. Ure, and J. Vian (2015) Optimized airborne collision avoidance. In Decision Making Under Uncertainty: Theory and Application, pp. 249–276. Cited by: §1.
  • [12] M. J. Kochenderfer and J. P. Chryssanthacopoulos (2010) A decision-theoretic approach to developing robust collision avoidance logic. In 13th International IEEE Conference on Intelligent Transportation Systems, pp. 1837–1842. External Links: Document Cited by: §1.
  • [13] M. J. Kochenderfer and N. Monath (2013) Compression of optimal value functions for markov decision processes. In 2013 Data Compression Conference, pp. 501–501. External Links: Document Cited by: §1.
  • [14] C. Liu, T. Arnon, C. Lazarus, C. Strong, C. Barrett, and M. J. Kochenderfer (2019) Algorithms for verifying deep neural networks. arXiv preprint arXiv:1903.06758. Cited by: §1.
  • [15] G. Singh, T. Gehr, M. Püschel, and M. T. Vechev (2019) Boosting robustness certification of neural networks.. In ICLR (Poster), Cited by: §1.
  • [16] G. Singh, T. Gehr, M. Püschel, and M. Vechev (2019) Boosting robustness certification of neural networks. In International Conference on Learning Representations, Cited by: §3.1.
  • [17] H. Tran, X. Yang, D. Manzanas Lopez, P. Musau, L. V. Nguyen, W. Xiang, S. Bak, and T. T. Johnson (2020) NNV: the neural network verification tool for deep neural networks and learning-enabled cyber-physical systems. In Computer Aided Verification, S. K. Lahiri and C. Wang (Eds.), Cham, pp. 3–17. External Links: ISBN 978-3-030-53288-8 Cited by: §1.
  • [18] C. Urban and A. Miné (2021)

    A review of formal methods applied to machine learning

    .
    arXiv preprint arXiv:2104.02466. Cited by: §1.
  • [19] S. Wang, K. Pei, J. Whitehouse, J. Yang, and S. Jana (2018) Formal security analysis of neural networks using symbolic intervals. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1599–1614. Cited by: item 2, item 3, §1, §3.1, §3.1, §3.2, Table 3.
  • [20] M. Wu and M. Kwiatkowska (2020) Robustness guarantees for deep neural networks on videos.

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    , pp. 308–317.
    Cited by: §2.