Bayesian Optimized 1-Bit CNNs

by   Jiaxin Gu, et al.

Deep convolutional neural networks (DCNNs) have dominated the recent developments in computer vision through making various record-breaking models. However, it is still a great challenge to achieve powerful DCNNs in resource-limited environments, such as on embedded devices and smart phones. Researchers have realized that 1-bit CNNs can be one feasible solution to resolve the issue; however, they are baffled by the inferior performance compared to the full-precision DCNNs. In this paper, we propose a novel approach, called Bayesian optimized 1-bit CNNs (denoted as BONNs), taking the advantage of Bayesian learning, a well-established strategy for hard problems, to significantly improve the performance of extreme 1-bit CNNs. We incorporate the prior distributions of full-precision kernels and features into the Bayesian framework to construct 1-bit CNNs in an end-to-end manner, which have not been considered in any previous related methods. The Bayesian losses are achieved with a theoretical support to optimize the network simultaneously in both continuous and discrete spaces, aggregating different losses jointly to improve the model capacity. Extensive experiments on the ImageNet and CIFAR datasets show that BONNs achieve the best classification performance compared to state-of-the-art 1-bit CNNs.


Projection Convolutional Neural Networks for 1-bit CNNs via Discrete Back Propagation

The advancement of deep convolutional neural networks (DCNNs) has driven...

Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagation

The rapidly decreasing computation and memory cost has recently driven t...

GBCNs: Genetic Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

Training 1-bit deep convolutional neural networks (DCNNs) is one of the ...

From Hashing to CNNs: Training BinaryWeight Networks via Hashing

Deep convolutional neural networks (CNNs) have shown appealing performan...

RedBit: An End-to-End Flexible Framework for Evaluating the Accuracy of Quantized CNNs

In recent years, Convolutional Neural Networks (CNNs) have become the st...

DenseShift: Towards Accurate and Transferable Low-Bit Shift Network

Deploying deep neural networks on low-resource edge devices is challengi...

Please sign up or login with your details

Forgot password? Click here to reset