Training Compact Neural Networks with Binary Weights and Low Precision Activations

08/08/2018
by   Bohan Zhuang, et al.
0

In this paper, we propose to train a network with binary weights and low-bitwidth activations, designed especially for mobile devices with limited power consumption. Most previous works on quantizing CNNs uncritically assume the same architecture, though with reduced precision. However, we take the view that for best performance it is possible (and even likely) that a different architecture may be better suited to dealing with low precision weights and activations. Specifically, we propose a "network expansion" strategy in which we aggregate a set of homogeneous low-precision branches to implicitly reconstruct the full-precision intermediate feature maps. Moreover, we also propose a group-wise feature approximation strategy which is very flexible and highly accurate. Experiments on ImageNet classification tasks demonstrate the superior performance of the proposed model, named Group-Net, over various popular architectures. In particular, with binary weights and activations, we outperform the previous best binary neural network in terms of accuracy as well as saving more than 5 times computational complexity on ImageNet with ResNet-18 and ResNet-50.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/22/2018

Rethinking Binary Neural Network for Accurate Image Classification and Semantic Segmentation

In this paper, we propose to train a network with both binary weights an...
research
01/16/2020

MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?

Binary Neural Networks (BNNs) are neural networks which use binary weigh...
research
09/26/2019

Balanced Binary Neural Networks with Gated Residual

Binary neural networks have attracted numerous attention in recent years...
research
03/16/2021

Training Dynamical Binary Neural Networks with Equilibrium Propagation

Equilibrium Propagation (EP) is an algorithm intrinsically adapted to th...
research
11/22/2018

Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation

In this paper, we propose to train convolutional neural networks (CNNs) ...
research
09/22/2019

Structured Binary Neural Networks for Image Recognition

We propose methods to train convolutional neural networks (CNNs) with bo...
research
04/04/2022

Soft Threshold Ternary Networks

Large neural networks are difficult to deploy on mobile devices because ...

Please sign up or login with your details

Forgot password? Click here to reset