Self-Distribution Binary Neural Networks

03/03/2021
by   Ping Xue, et al.
0

In this work, we study the binary neural networks (BNNs) of which both the weights and activations are binary (i.e., 1-bit representation). Feature representation is critical for deep neural networks, while in BNNs, the features only differ in signs. Prior work introduces scaling factors into binary weights and activations to reduce the quantization error and effectively improves the classification accuracy of BNNs. However, the scaling factors not only increase the computational complexity of networks, but also make no sense to the signs of binary features. To this end, Self-Distribution Binary Neural Network (SD-BNN) is proposed. Firstly, we utilize Activation Self Distribution (ASD) to adaptively adjust the sign distribution of activations, thereby improve the sign differences of the outputs of the convolution. Secondly, we adjust the sign distribution of weights through Weight Self Distribution (WSD) and then fine-tune the sign distribution of the outputs of the convolution. Extensive experiments on CIFAR-10 and ImageNet datasets with various network structures show that the proposed SD-BNN consistently outperforms the state-of-the-art (SOTA) BNNs (e.g., achieves 92.5 ImageNet with ResNet-18) with less computation cost. Code is available at https://github.com/ pingxue-hfut/SD-BNN.

READ FULL TEXT
research
08/17/2022

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

This paper studies the Binary Neural Networks (BNNs) in which weights an...
research
02/16/2021

SiMaN: Sign-to-Magnitude Network Binarization

Binary neural networks (BNNs) have attracted broad research interest due...
research
04/05/2022

Bimodal Distributed Binarized Neural Networks

Binary Neural Networks (BNNs) are an extremely promising method to reduc...
research
10/08/2021

Dynamic Binary Neural Network by learning channel-wise thresholds

Binary neural networks (BNNs) constrain weights and activations to +1 or...
research
11/04/2022

Boosting Binary Neural Networks via Dynamic Thresholds Learning

Developing lightweight Deep Convolutional Neural Networks (DCNNs) and Vi...
research
04/16/2019

Matrix and tensor decompositions for training binary neural networks

This paper is on improving the training of binary neural networks in whi...
research
07/19/2022

RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating

Binary neural network (BNN) is an extreme quantization version of convol...

Please sign up or login with your details

Forgot password? Click here to reset