IR2Net: Information Restriction and Information Recovery for Accurate Binary Neural Networks

by   Ping Xue, et al.

Weight and activation binarization can efficiently compress deep neural networks and accelerate model inference, but cause severe accuracy degradation. Existing optimization methods for binary neural networks (BNNs) focus on fitting full-precision networks to reduce quantization errors, and suffer from the trade-off between accuracy and computational complexity. In contrast, considering the limited learning ability and information loss caused by the limited representational capability of BNNs, we propose IR^2Net to stimulate the potential of BNNs and improve the network accuracy by restricting the input information and recovering the feature information, including: 1) information restriction: for a BNN, by evaluating the learning ability on the input information, discarding some of the information it cannot focus on, and limiting the amount of input information to match its learning ability; 2) information recovery: due to the information loss in forward propagation, the output feature information of the network is not enough to support accurate classification. By selecting some shallow feature maps with richer information, and fusing them with the final feature maps to recover the feature information. In addition, the computational cost is reduced by streamlining the information recovery method to strike a better trade-off between accuracy and efficiency. Experimental results demonstrate that our approach still achieves comparable accuracy even with ∼10x floating-point operations (FLOPs) reduction for ResNet-18. The models and code are available at


page 1

page 9


MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy?

Binary Neural Networks (BNNs) are neural networks which use binary weigh...

Bimodal Distributed Binarized Neural Networks

Binary Neural Networks (BNNs) are an extremely promising method to reduc...

IR-Net: Forward and Backward Information Retention for Highly Accurate Binary Neural Networks

Weight and activation binarization is an effective approach to deep neur...

RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating

Binary neural network (BNN) is an extreme quantization version of convol...

Searching Similarity Measure for Binarized Neural Networks

Being a promising model to be deployed in resource-limited devices, Bina...

Any-Precision Deep Neural Networks

We present Any-Precision Deep Neural Networks (Any-Precision DNNs), whic...

INSTA-BNN: Binary Neural Network with INSTAnce-aware Threshold

Binary Neural Networks (BNNs) have emerged as a promising solution for r...

Please sign up or login with your details

Forgot password? Click here to reset