Binarized Weight Error Networks With a Transition Regularization Term

05/09/2021
by   Savas Ozkan, et al.
0

This paper proposes a novel binarized weight network (BT) for a resource-efficient neural structure. The proposed model estimates a binary representation of weights by taking into account the approximation error with an additional term. This model increases representation capacity and stability, particularly for shallow networks, while the computation load is theoretically reduced. In addition, a novel regularization term is introduced that is suitable for all threshold-based binary precision networks. This term penalizes the trainable parameters that are far from the thresholds at which binary transitions occur. This step promotes a swift modification for binary-precision responses at train time. The experimental results are carried out for two sets of tasks: visual classification and visual inverse problems. Benchmarks for Cifar10, SVHN, Fashion, ImageNet2012, Set5, Set14, Urban and BSD100 datasets show that our method outperforms all counterparts with binary precision.

READ FULL TEXT
research
12/31/2018

BNN+: Improved Binary Network Training

Deep neural networks (DNN) are widely used in many applications. However...
research
03/08/2018

Learning Effective Binary Visual Representations with Deep Networks

Although traditionally binary visual representations are mainly designed...
research
04/05/2022

Bimodal Distributed Binarized Neural Networks

Binary Neural Networks (BNNs) are an extremely promising method to reduc...
research
06/23/2023

Binary domain generalization for sparsifying binary neural networks

Binary neural networks (BNNs) are an attractive solution for developing ...
research
07/03/2021

Exact Backpropagation in Binary Weighted Networks with Group Weight Transformations

Quantization based model compression serves as high performing and fast ...
research
11/07/2018

Median Binary-Connect Method and a Binary Convolutional Neural Nework for Word Recognition

We propose and study a new projection formula for training binary weight...
research
11/04/2022

Boosting Binary Neural Networks via Dynamic Thresholds Learning

Developing lightweight Deep Convolutional Neural Networks (DCNNs) and Vi...

Please sign up or login with your details

Forgot password? Click here to reset