Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

12/23/2019
by   Sébastien Henwood, et al.
0

Deep neural networks (DNNs) depend on the storage of a large number of parameters, which consumes an important portion of the energy used during inference. This paper considers the case where the energy usage of memory elements can be reduced at the cost of reduced reliability. A training algorithm is proposed to optimize the reliability of the storage separately for each layer of the network, while incurring a negligible complexity overhead compared to a conventional stochastic gradient descent training. For an exponential energy-reliability model, the proposed training approach can decrease the memory energy consumption of a DNN with binary parameters by 3.3× at isoaccuracy, compared to a reliable implementation.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

11/23/2019

Training Modern Deep Neural Networks for Memory-Fault Robustness

Because deep neural networks (DNNs) rely on a large number of parameters...
12/02/2017

LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks

Application-specific integrated circuit (ASIC) implementations for Deep ...
09/06/2020

TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training

Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs...
07/23/2018

NullaNet: Training Deep Neural Networks for Reduced-Memory-Access Inference

Deep neural networks have been successfully deployed in a wide variety o...
05/08/2020

Efficient Computation Reduction in Bayesian Neural Networks Through Feature Decomposition and Memorization

Bayesian method is capable of capturing real world uncertainties/incompl...
01/26/2013

Neural Networks Built from Unreliable Components

Recent advances in associative memory design through strutured pattern s...
06/01/2021

More Behind Your Electricity Bill: a Dual-DNN Approach to Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) is a well-known single-channel blin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.