Layerwise Noise Maximisation to Train Low-Energy Deep Neural Networks

by   Sébastien Henwood, et al.

Deep neural networks (DNNs) depend on the storage of a large number of parameters, which consumes an important portion of the energy used during inference. This paper considers the case where the energy usage of memory elements can be reduced at the cost of reduced reliability. A training algorithm is proposed to optimize the reliability of the storage separately for each layer of the network, while incurring a negligible complexity overhead compared to a conventional stochastic gradient descent training. For an exponential energy-reliability model, the proposed training approach can decrease the memory energy consumption of a DNN with binary parameters by 3.3× at isoaccuracy, compared to a reliable implementation.



page 1

page 2

page 3

page 4


Training Modern Deep Neural Networks for Memory-Fault Robustness

Because deep neural networks (DNNs) rely on a large number of parameters...

LightNN: Filling the Gap between Conventional Deep Neural Networks and Binarized Networks

Application-specific integrated circuit (ASIC) implementations for Deep ...

TaxoNN: A Light-Weight Accelerator for Deep Neural Network Training

Emerging intelligent embedded devices rely on Deep Neural Networks (DNNs...

NullaNet: Training Deep Neural Networks for Reduced-Memory-Access Inference

Deep neural networks have been successfully deployed in a wide variety o...

Efficient Computation Reduction in Bayesian Neural Networks Through Feature Decomposition and Memorization

Bayesian method is capable of capturing real world uncertainties/incompl...

Neural Networks Built from Unreliable Components

Recent advances in associative memory design through strutured pattern s...

More Behind Your Electricity Bill: a Dual-DNN Approach to Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) is a well-known single-channel blin...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.