Sigma Delta Quantized Networks

11/07/2016
by   Peter O'Connor, et al.
0

Deep neural networks can be obscenely wasteful. When processing video, a convolutional network expends a fixed amount of computation for each frame with no regard to the similarity between neighbouring frames. As a result, it ends up repeatedly doing very similar computations. To put an end to such waste, we introduce Sigma-Delta networks. With each new input, each layer in this network sends a discretized form of its change in activation to the next layer. Thus the amount of computation that the network does scales with the amount of change in the input and layer activations, rather than the size of the network. We introduce an optimization method for converting any pre-trained deep network into an optimally efficient Sigma-Delta network, and show that our algorithm, if run on the appropriate hardware, could cut at least an order of magnitude from the computational cost of processing video data.

READ FULL TEXT

page 7

page 8

research
07/30/2018

Multi-Fiber Networks for Video Recognition

In this paper, we aim to reduce the computational cost of spatio-tempora...
research
06/22/2015

Understanding Neural Networks Through Deep Visualization

Recent years have produced great advances in training large, deep neural...
research
01/01/2018

Depth-Adaptive Computational Policies for Efficient Visual Tracking

Current convolutional neural networks algorithms for video object tracki...
research
07/15/2021

Training for temporal sparsity in deep neural networks, application in video processing

Activation sparsity improves compute efficiency and resource utilization...
research
12/29/2019

MTJ-Based Hardware Synapse Design for Quantized Deep Neural Networks

Quantized neural networks (QNNs) are being actively researched as a solu...
research
05/10/2023

Treasure What You Have: Exploiting Similarity in Deep Neural Networks for Efficient Video Processing

Deep learning has enabled various Internet of Things (IoT) applications....
research
09/21/2023

Video Scene Location Recognition with Neural Networks

This paper provides an insight into the possibility of scene recognition...

Please sign up or login with your details

Forgot password? Click here to reset