With the widespread use of convolutional neural networks (CNN) across many safety-critical domains such as automated robots and cars, one of the most prevailing challenges is the establishment of a safety certification for such artificial intelligence (AI) components, e.g., with respect to the ISO 26262[Iso26262] or ISO/PAS 21448 (SOTIF) [Sotif2019]. This certification requires not only a high fault-tolerance of the trained network against unknown or adversarial input, but also efficient protection against hardware faults of the underlying platform [Athavale2020, Dixit2021]. Importantly this includes transient soft errors, meaning disturbances originating from events such as cosmic neutron radiation, isotopes emitting alpha particles, or electromagnetic leakage on the computer circuitry itself.
Soft errors typically manifest as single or multiple bit upsets in the platform’s memory elements [Neale2016]. As a consequence, network parameters (weight faults) or local computational states (neuron faults) can be altered during inference time, and invalidate the network prediction in a safety-critical way, for example, by misclassifying a person as a background image in an automated driving context [Li2017, hoang2019ftclipact, chen2020ranger]. This has led to a search for strategies to verify CNN-based systems against hardware faults at the inference stage [Cluzeau2020]. With chip technology nodes scaling to smaller sizes and larger memory density per area, future platforms are expected to be even more susceptible to soft errors [Neale2016].
In this paper, we evaluate range restriction techniques in CNNs exposed to platform soft errors with respect to the key elements of a prototypical safety case. This means that we formulate arguments (in the form of ”goals”) that constitute essential parts of a complete safety case, and provide quantitative evidence to support these goals in the studied context (see Fig. 1). Individual safety arguments can be reused as building blocks of more complex safety cases. The structure of our goals is based on the probabilistic, high-level safety objective of minimizing the overall risk [Koopman2019], expressed as:
Explicitly, for a fault type , this includes the sub-goals of efficient error detection and mitigation, as well as a consideration of the fault severity in a given use case. On the other hand, the probability of occurrence of a soft error (i.e., in Eq. 1) is assumed to be a constant system property that cannot be controlled by run-time monitoring methods such as activation range supervision.
In a nutshell, range restriction builds on the observation that silent data corruption (SDC) and detected uncorrectable errors (DUE), e.g., NaN and Inf occurrences), stem primarily from those bit flips that cause very large values, for example in high exponential bits [Li2017]. Those events result in large activation peaks that typically grow even more during forward propagation due to the monotonicity of most neural network operations [Chen2019a]. To suppress the propagation of such corrupted values, additional range restriction layers are inserted in the network at strategic positions following the approach of Chen et. al. [chen2020ranger] (see Fig. 2 for an example). At inference time, the protection layers then compare the intermediate activations against previously extracted interval thresholds in order to detect and reset anomalously large values. Derivative approaches have been shown to be efficient in recovering network performance [Li2017, Hong2019, chen2020ranger, hoang2019ftclipact] and, advantageously, do not require the retraining of CNN parameters nor computationally expensive functional duplications.
The focus of this paper is to examine alternative restriction schemes for optimized soft error mitigation. In a CNN, the output of every kernel is represented as a two-dimensional (2D) feature map, where the activation magnitudes encode specific features, on which the network bases its prediction. Soft errors will manifest as distortions of feature maps in all subsequent layers that make use of the corrupted value, as shown in Fig. 2(a)-(b). The problem of mitigating soft errors in a CNN can therefore be rephrased as restoring the fault-free topology of feature maps.
Previous analyses have adopted uniform range restriction schemes that truncate out-of-bound values to a finite threshold [chen2020ranger, hoang2019ftclipact], e.g., Fig. 2(c)-(d). We instead follow the intuition that optimized, non-uniform range restriction methods that attempt to reconstruct feature maps (see Fig. 2(e)-(g), and details in Sec. 5) can not only reduce SDC to a comparable or even lower level, but may also lead to less critical misclassifications in the case of an SDC. This is because classes with more similar attributes will display more similar high-level features (e.g., pedestrian and biker will both exhibit upright silhouette, in contrast to car and truck classes).
Finally, a safety analysis has to consider that not all SDC events pose an equal risk to the user. We study a safety-critical use case evaluating cluster-wise class confusions in a vehicle classification scenario (Sec. 6). The example shows that range supervision reduces the severe confusions proportionally with the overall number of confusions, meaning that the total risk is indeed mitigated.
In summary, this paper make the following contributions:
Fault detection: We quantify the correlation between SDC events and the occurrence of out-of-bound activations to demonstrate the high efficiency of fault detection by monitoring intermediate activations,
Fault mitigation: We explore three novel range restriction methods that build on the preservation of the feature map topologies instead of mere value truncation,
We demonstrate the benefit of range supervision in an end-to-end use case of vehicle classification where high and low severities are estimated by the generic safety-criticality of class confusions.
The article is structured as follows: Section 2 reviews relevant previous work while section 3 describes the setup used in this paper. Subsequently, the sections 4, 5, and 6 discuss error detection, mitigation, and an exemplary risk analysis, respectively, before section 7 concludes the paper.
2 Related work
Parity or error-correcting code (ECC) can protect memory elements against single soft errors [Neale2016, Lotfi2019]. However, due to the high compute and area overhead, this is typically done only for selected critical memory blocks. Component replication techniques such as triple modular redundancy can be used for the full CNN execution at the cost of a large overhead. Selective hardening of hardware elements with the most salient parameters can improve the robustness of program execution in the presence of underlying faults [Li2017, Hanif2020]. On a software level, the estimation of the CNN’s vulnerable feature maps (fmaps) and the selective protection by duplicated computations [mahmoud2020hardnn], or the assertive re-execution with stored, healthy reference values [Ponader2020] has been investigated. Approaches using algorithm-based fault tolerance (ABFT) [Zhao2021] seek to protect networks against soft errors by checking invariants that are characteristic for a specific operation (e.g., matrix multiplication). Symptom-based error detection may for example include the interpretation of feature map traces by a secondary companion network [Schorn2018]. The restriction of intermediate ranges was explored [Li2017, Hong2019]
in the form of modified (layer-insensitive) activation functions such asor . This concept was extended to find specific uniform protection thresholds for neuron faults [chen2020ranger] or clipping bounds for weight faults [hoang2019ftclipact]. An alternative line of research is centered around fault-aware retraining [Yang2017].
3 Experimental setup
3.1 Models, data sets, and system
CNNs are the most commonly used network variant for computer vision tasks such as object classification and detection. We compare the three standard classifier CNNs ResNet-50 [He2016], VGG-16 [Simonyan2015], and AlexNet [Krizhevsky2012]
together with the test data set ImageNet[JiaDeng2009] and MIOVision [Luo2018]
for the investigation of a safety-critical example use case. Since fault injection is compute-intensive, we rescale our test data set for ImageNet to a subset ofimages representing randomly selected classes. For MIOVision, a subset of images ( per class) that were correctly classified in the absence of faults was chosen. All experiments adopt a single-precision floating point format (FP32) according to the IEEE754 standard [IEEE2019]. Our conclusions apply as well to other floating point formats with the same number of exponent bits, such as BF16 [Intel2018], since no relevant effect was observed from fault injections in mantissa bits (Sec. 4).
Experiments were performed in PyTorch (version 1.8.0) deploying torchvision models (version 0.9.0). For MIOVision, the ResNet-50 model was retrained[Theagarajan2017]. We used Intel Core i9 CPUs, with inferences running on GeForce RTX , Titan RTX, and RTX 3090 GPUs.
3.2 Protection layers and bound extraction
We insert protection layers at strategic positions in the network such as after activation, pooling, reshape or concatenate layers, according to the model of Chen et al. [chen2020ranger]. Each protection layer requires specific bound values for the expected activation ranges as a parameter. We extract those by monitoring the minimal and maximal activations from a separate test input, which is taken from the training data sets of ImageNet ( images used) and MIOVision ( images used), respectively. This step has to be performed only once. Bound extraction depends on the data set and will in general impact the safety argument (see Fig. 1). To check the suitability of the bounds, we verify that no out-of-bound events were detected during the test phase in the absence of faults, so the baseline accuracy is the same with and without protection. While all minimum bounds are zero in the studied setup, the maximum activation values for ImageNet vary by layer in a range of (see also Sec. 5) for ResNet-50, for VGG-16, and for AlexNet. For MIOVision and ResNet-50, we find maximum bounds between .
3.3 Fault model and injection
In line with previous investigations, we distinguish two different manifestations of memory bit flips referred to here as weight faults and neuron faults. The former represent soft errors affecting memory elements that store the learned network parameters, while the latter refer to errors in memory that holds temporary states such as intermediate network layer outputs. While neuron faults may also impact states used for logical instructions, it was demonstrated that bit flip injections in the output of the affected layer are generally a good model approximation [Chang2019]. Memory elements can be protected against single bit flips by mechanisms such as parity and ECC [Neale2016, Lotfi2019]. However, this kind of protection is not always available due to the associated compute and area overhead. Further, ECC typically cannot correct multi-bit flips.
We inject faults either directly in the weights of CNN layers (weight faults) or in the output of the latter (neuron faults), using a customized fault injection framework based on PytorchFI [Mahmoud2020a]. To speed up the experiments we focus on bit flips in the most relevant bit positions
(sign bit and exponential bits, neglecting mantissa) unless stated otherwise. Fault locations (i.e., layer index, kernel index, channel etc.) in the network are randomly chosen with an equal weight, so without further constraints on the selection process to reflect the arbitrary occurrence of soft errors. As weights are typically stored in the main memory and loaded only once for a given application, we keep the same weight faults for one entire epoch, running all tested input images. In total, we runepochs, i.e., fault configurations, each one applied to images. Neuron faults, on the other hand, apply to memory representing temporary states that are overwritten for each new input. Therefore, we inject new neuron faults for each new input and run epochs resulting in fault configurations, each one applied to a single image.
To quantify the impact of faults on the system safety, we measure the rate of SDC events. Throughout, we consider the Top-1 prediction to determine SDC. In line with previous work [chen2020ranger, Li2017], SDC is defined as the ratio of images that are misclassified in the presence of faults (without exceptions) but correctly classified in the absence of faults and the overall number of images, (Fig. 3).
During the forward pass, non-numerical exceptions in the form of Inf and NaN values can be encountered, due to the following reasons: i) Inf values occur if large activation values accumulate (for example during conv2d, linear, avgpool2d operations) until they exceed the maximum of the data representation. This effect becomes particularly apparent when flips of the most significant bit (MSB, position index ) are injected. ii) NaN values are found when denominators are undetermined or multiple Inf values get added, e.g., in BatchNorm2d layers, iii) NaN values can be generated directly via bit flips in conv2d layers due to the fact that FP32 encodes NaN as all eight exponent bits being in state . In the studied classifier networks, the latter effect is very rare for single bit flips in weights (see Sec. 4) but not necessarily for single neuron bit flips or multiple flips of either type.
The creation of the above exceptions is found to differ slightly between CPU and GPU executions, as well as between experiments with different batch sizes on the accelerator. We attribute this observation to algorithmic optimizations on the GPU that are not necessarily IEEE754-compliant and thus affect the floating point precision [nvidia2021]. To mitigate the effect of exception handling we monitor the occurrences of Inf and NaN in the output of any network layer. All forward passes with an exception are separated and define the detected uncorrectable error (DUE) rate, , see Fig. 3.
In a real system, DUE events can be readily monitored and the execution is typically halted on detection. However, due to the non-numerical nature of these errors we cannot apply the same mitigation strategy that is adopted for SDC events. We therefore make the assumption that either a fallback system (e.g., alternative classifier, emergency stop of vehicle, etc.) can be leveraged or a timely re-execution is possible to recover from transient DUE events. This in turn assumes that DUEs do not impact the system safety but may compromise the system availability when occurring frequently.
4 Error detection coverage
To effectively protect the network against faults, we first verify the error detection coverage for silent errors. Those errors are detected by a given protection layer if the activation values exceed (fall short of) the upper (lower) bound. If at least one protection layer is triggered per inference run, we register an out-of-bound (oob) event. Otherwise, we have an in-bound (ib) event. In addition, we quantify the probabilities of SDC and regular correct classification events, as well as the respective conditional probabilities that correct and incorrect classifications occur given that oob or ib events were detected. This allows us to define true positive (Tp), false positive (Fp), and false negative (Fn) SDC detection rates as
The fault detector then is characterized by precision, , and recall, .
The Tab. 1 displays the chances of oob and sdc events resulting from a single fault per image in the absence of range protection. For weight faults, we find that all three CNNs showcase a high correlation between oob situations and either SDC or DUE events (), which can be associated with the chance of a successful error detection, (see Eq. 1). The chance of finding SDC after ib events is very small (
), leading to a very high precision and recall performance (). For neuron faults, while the recall remains very high, the precision is reduced (in particular VGG-16 and AlexNet) due to additional Fp events where non-MSB oob events still get classified correctly.
We further verify that SDC events from single weight faults are attributed almost exclusively to flips of the MSB. This can be explained with the distribution of parameters in the studied networks (Fig. 4). The weight values are closely centered around zero, and thus exhibit characteristic properties when represented in an eight-exponent data format. In the fault-free case, the MSB always has state “0”, while the exponent bits to are almost always in state “1”. This means that among the relevant exponential bits all single bit flips of the MSB will produce large values, while those of the other exponential bits will either be from or will be too small to have a significant effect.
For neuron faults, on the other hand, the distribution of fault-free values is input-dependent and broader, leading in general to a smaller quota of MSB flips to SDC, in favor of flips of other exponential bits and the sign bit. No SDC due to mantissa bit flips were observed in either weight or neuron faults. DUE events are unlikely () for a single bit flip as there are not multiple large values to add up. Further, network weights are usually , meaning that at least two exponent bits are in state , and hence at least two bit flips are needed to directly generate a NaN value.
|Weight faults||Neuron faults|
5 Range restriction methods for error mitigation
We refer to a subset of the tensor given by a specific index in the batch and channel dimensions as a 2D feature map, denoted by. Let be an activation value from a given feature map tensor . Further, and denote the upper and lower activation bounds assigned to the protection layer, respectively.
Ranger: For a given set of , Ranger [chen2020ranger] maps out-of-bound values to the expected interval (see Fig. 2c),
Clipper: In a similar way, clipping truncates activations that are out of bound to zero [hoang2019ftclipact],
The intuition is that it can be favorable to eliminate corrupted elements rather than to re-establish finite activations.
FmapRescale: While uniform restriction methods help in eliminating large out-of-bound values, the information encoded in relative differences of activation magnitudes is lost when all out-of-bound values are flattened to the same value. The idea of rescaling is to linearly map all large out-of-bound values back onto the interval , implying that smaller out-of-bound values are reduced more. This follows the intuition that the out-of-bound values can originate from the entire spectrum of in-bound values.
Backflip: We analyze the underlying bit flips that may have caused out-of-bound values. This reasoning holds for neuronal faults, where we may assume that a specific activation value is bit-flipped directly. For weight faults, on the other hand, the observed out-of-bound output activation is the result of a multiply-and-accumulate operation of an input tensor with a bit-flipped weight value. However, we argue that the presented back-flip operation will recover a representative product, given that the input component is of the order of magnitude of one. To restore a flipped value, we distinguish the following cases:
The above thresholds are motivated by the following logic: Given appropriate bounds, an activation is before a bit flip. Any flip of an exponential bit effectively multiplies a factor of . Hence, any value beyond must have originated from a flip of the MSB, meaning that the original value was between and . We then set back all out-of-bound values in this regime to zero, assuming that lower reset values represent a more conservative choice in eliminating faults. Next, flipped values that are between can possibly originate from a flip of any exponential bit. Given that is typically , a bit flip has to produce a corrupted absolute value in this regime. This is possible only if either the MSB is flipped from , or the MSB is already at and another exponential bit is flipped . In all variants of the latter case, the original value had to be already itself, and hence we conservatively reset out-of-bound values to . Finally, corrupted values of may originate from any non-sign bit flip. Lower exponential or even fraction bit flips result from already large values close to in this case, which is why we set back those values to the upper bound. As in Ranger, values that are too small are reset to .
FmapAvg: The last proposed range restriction technique uses the remaining, healthy fmaps of a convolutional layer to reconstruct a corrupted fmap. The intuition behind this approach is as follows: Every filter in a given conv2d layer tries to establish characteristic features of the input image. Typically, there is a certain redundancy in the topology of fmaps, since not all features the network was trained to recognize may be strongly pronounced for a given image (instead mixtures of potential features may form), or because multiple features resemble each other at the given processing stage. Therefore, replacing a corrupted fmap with a non-corrupted fmap from a different kernel can help to obtain an estimate of the original topology. We average all healthy (i.e., not containing out-of-bound activations) fmaps by
If there are no healthy feature maps, will be the zero-tensor. Subsequently, we replace oob values in a corrupted fmap with their counterparts from the estimate of Eq. (7),
In Fig. 5 we present results for the SDC mitigation experiments with different range supervision methods. Comparing and fault injections per input image, we note that the unprotected models are dramatically corrupted with an increasing fault rate (SDC rate becomes for weights, for neurons in the presence of faults). We can associate the SDC rate with the chance of unsuccessful mitigation, , in Eq. 1. Weight faults have a higher impact than neuron faults since they directly corrupt a multitude of activations in a layer’s fmap output (in contrast to individual activations for neuron faults) and thus propagate faster than neuron faults.
All the studied range restriction methods reduce the SDC rate by a significant margin, but perform differently for weight and neuron fault types: For weight faults, we observe that Clipper, Backflip, and FmapAvg are highly efficient in all three networks, with SDC rates suppressed to values of (SDC reduction of ). Ranger provides a much weaker protection, in particular in the more shallow networks VGG-16 and AlexNet. FmapRescale performs better than Ranger but worse than the aforementioned methods. The deepest studied network, ResNet-50, benefits the most from any type of range restriction in the presence of weight faults.
When it comes to neuron faults (Fig. 5b), we see that Clipper and Backflip provide the best protection (SDC rate is suppressed to , reduction of ), followed by the also very effective Ranger (except for AlexNet). FmapAvg appears to be less efficient for higher fault rates in this scenario, while FmapRescale again falls behind all the above.
Overall, we conclude that the pruning-inspired mitigation techniques Clipper and Backflip represent the best choices among the investigated ranger supervision methods, as they succeed in mitigating both weight and neuron faults to very small residual SDC rates.
In the experiments of Fig. 5, the encountered DUE rates for weight or neuron fault ( for ResNet, for VGG-16 or AlexNet) are only slightly reduced by range restrictions. However, for a fault rate of we find the following trends: i) For weights, the DUE is significantly reduced in ResNet (from to ), while rates in VGG () and AlexNet () remain. ii) For neurons, Ranger, Clipper and Backflip suppress the DUE rate by a factor of up to in all networks.
The studied range restriction techniques require different compute costs due to the different number of additional graph operations. In PyTorch, not all needed functions can be implemented with the same efficiency though. For example, Ranger is executed with a single clamp operation, while no equivalent formulation is available for Clipper and instead three operations are necessary (two masks to select oob values greater and smaller than the threshold, and a masked-fill operation to clip to zero). As a consequence, measured latencies are framework-dependent and a fair comparison cannot be made at this point. Given the complexity of the protection operations, we may instead give a qualitative performance ranking of the described methods: FmapRescale appears to be the most expensive restriction method due to the needed number of operations, followed by FmapAvg and Backflip. Clipper and Ranger are the least complex, with the latter outperforming the former in the used framework, due to its more efficient use of optimized built-in operations.
6 Analysis of traffic camera use case
As a selected safety-critical use case, we study object classification in the presence of soft errors with a retrained ResNet-50 and the MIOVision data set [Luo2018]. The data contains images of classes including for example pedestrian, bike, car, or background, that were taken by traffic cameras. The correct identification of an object type or category can be safety-critical for example to an automated vehicle that uses the support of infrastructure sensors for augmented perception [Krammer2019].
However, not every class confusion is equally harmful. To estimate the severity of an error-induced misclassification we establish three clusters of vulnerable, as well as non-vulnerable road users (VRU or non-VRU), and background, see Fig. 6. Misclassifications that lead to the prediction of a class in a less vulnerable cluster are assumed to be safety-critical ( in Eq. 1, e.g., a pedestrian is misclassified as background), while confusions within the same cluster or towards a more vulnerable cluster are considered non-critical () as they typically lead only to similar or a more cautious behavior. This binary estimation allows us quantify the overall risk as the portion of SDC events associated with the respective critical class confusions.
From our results in Fig. 7 we make the following observations: i) The relative proportion of critical confusions is lower for weight than for neuron faults in the unprotected and most protected models. For weight faults, the most frequent confusions are from other classes to the class ”car” (the most robust class of MIOVision, with the most images in the training set), which are statistically mostly non-critical. Neuron faults, on the other hand, distort feature maps in a way that induces with the highest frequency misclassifications towards the class ”background”. Those events are all safety-critical (see Fig. 6), leading to a high critical-to-total SDC ratio. ii) Range supervision is not only effective in reducing the overall SDC count, but also suppresses the critical SDC count proportionally. For example, we observe that the most frequent critical class confusion caused by or weight faults is from the class ”pedestrian” to ”car” ( of all critical SDC cases), where of those cases can be mitigated by Clipper or Backflip. For neuron faults, the largest critical SDC contribution is from ”pedestrian” to ”background” ( fault) or ”car” to ”background” ( faults), both in about of all critical SDC cases. Clipper or Backflip are able to suppress of those events.
As a consequence, all studied range-restricted models exhibit a critical-to-total SDC ratio that is similar to or lower than one of the unprotected network ( for weight, for neuron faults), meaning that faults in the presence of range supervision have on average a similar or lower severity than faults that do not face range restrictions. A lower ratio can be interpreted as a better preservation of the feature map topology: If the reconstructed features are more similar to the original features there is a higher chance of the incorrect class being similar to the original class and thus to stay within the same cluster. The total probability of critical SDC events – and therefore the relative risk according to Eq. 1 – is negligible in the studied setup in the presence of Clipper or Backflip range protection.
The mean DUE rates in the unprotected model are () for weight (neuron) fault and () for faults. Using any of the protection methods, the system’s availability increases as DUE rates are negligible for fault, and reduce to () for weight (neuron) faults.
In this paper, we investigated the efficacy of range supervision techniques for constructing a safety case for computer vision AI applications that use Convolutional Neural Networks (CNNs) in the presence of platform soft errors. In the given experimental setup, we demonstrated that the implementation of activation bounds allows for a highly efficient detection of SDC-inducing faults, most importantly featuring a recall of . Furthermore, we found that the range restriction layers can mitigate the once-detected faults effectively by mapping out-of-bound values back to the expected intervals. Exploring distinct restriction methods, we observed that Clipper and Backflip perform best for both weight and neuron faults and can reduce the residual SDC rate to (reduction by a factor of ). Finally, we studied the selected use case of vehicle classification to quantify the impact of range restriction on the severity of SDC events (represented by cluster-wise class confusions). All discussed techniques reduce critical and non-critical events proportionally, meaning that the average severity of SDC is not increased. Therefore, we conclude that the presented approach reduces the overall risk and thus enhances the safety of the user in the presence of platform soft errors.
Our research was partially funded by the Federal Ministry of Transport and Digital Infrastructure of Germany in the project Providentia++ (01MM19008). Further, this research was partially supported by a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC), and a research gift from Intel to UBC.