Controlling Information Capacity of Binary Neural Network

08/04/2020
by   Dmitry Ignatov, et al.
53

Despite the growing popularity of deep learning technologies, high memory requirements and power consumption are essentially limiting their application in mobile and IoT areas. While binary convolutional networks can alleviate these problems, the limited bitwidth of weights is often leading to significant degradation of prediction accuracy. In this paper, we present a method for training binary networks that maintains a stable predefined level of their information capacity throughout the training process by applying Shannon entropy based penalty to convolutional filters. The results of experiments conducted on SVHN, CIFAR and ImageNet datasets demonstrate that the proposed approach can statistically significantly improve the accuracy of binary networks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/30/2017

Towards Accurate Binary Convolutional Neural Network

We introduce a novel scheme to train binary convolutional neural network...
research
10/24/2019

Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagation

The rapidly decreasing computation and memory cost has recently driven t...
research
09/27/2018

Learning to Train a Binary Neural Network

Convolutional neural networks have achieved astonishing results in diffe...
research
03/25/2019

Active Learning of Spin Network Models

Complex networks can be modeled as a probabilistic graphical model, wher...
research
09/11/2020

SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks

Binary Convolutional Neural Networks (BCNNs) can significantly improve t...
research
11/25/2019

GBCNs: Genetic Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

Training 1-bit deep convolutional neural networks (DCNNs) is one of the ...
research
10/07/2020

High-Capacity Expert Binary Networks

Network binarization is a promising hardware-aware direction for creatin...

Please sign up or login with your details

Forgot password? Click here to reset