FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions

12/27/2021
by   Behnam Ghavami, et al.
0

Deep neural networks (DNNs) are increasingly being deployed in safety-critical systems such as personal healthcare devices and self-driving cars. In such DNN-based systems, error resilience is a top priority since faults in DNN inference could lead to mispredictions and safety hazards. For latency-critical DNN inference on resource-constrained edge devices, it is nontrivial to apply conventional redundancy-based fault tolerance techniques. In this paper, we propose FitAct, a low-cost approach to enhance the error resilience of DNNs by deploying fine-grained post-trainable activation functions. The main idea is to precisely bound the activation value of each individual neuron via neuron-wise bounded activation functions so that it could prevent fault propagation in the network. To avoid complex DNN model re-training, we propose to decouple the accuracy training and resilience training and develop a lightweight post-training phase to learn these activation functions with precise bound values. Experimental results on widely used DNN models such as AlexNet, VGG16, and ResNet50 demonstrate that FitAct outperforms state-of-the-art studies such as Clip-Act and Ranger in enhancing the DNN error resilience for a wide range of fault rates while adding manageable runtime and memory space overheads.

READ FULL TEXT
research
12/02/2019

FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation

Deep Neural Networks (DNNs) are widely being adopted for safety-critical...
research
01/08/2021

Exploring Fault-Energy Trade-offs in Approximate DNN Hardware Accelerators

Systolic array-based deep neural network (DNN) accelerators have recentl...
research
09/11/2020

Abstract Neural Networks

Deep Neural Networks (DNNs) are rapidly being applied to safety-critical...
research
11/24/2020

Provably-Robust Runtime Monitoring of Neuron Activation Patterns

For deep neural networks (DNNs) to be used in safety-critical autonomous...
research
06/16/2023

Enhancing Fault Resilience of QNNs by Selective Neuron Splitting

The superior performance of Deep Neural Networks (DNNs) has led to their...
research
09/30/2019

Automated design of error-resilient and hardware-efficient deep neural networks

Applying deep neural networks (DNNs) in mobile and safety-critical syste...
research
06/19/2023

Learn to Accumulate Evidence from All Training Samples: Theory and Practice

Evidential deep learning, built upon belief theory and subjective logic,...

Please sign up or login with your details

Forgot password? Click here to reset