Elastic-Link for Binarized Neural Network

12/19/2021
by   Jie Hu, et al.
0

Recent work has shown that Binarized Neural Networks (BNNs) are able to greatly reduce computational costs and memory footprints, facilitating model deployment on resource-constrained devices. However, in comparison to their full-precision counterparts, BNNs suffer from severe accuracy degradation. Research aiming to reduce this accuracy gap has thus far largely focused on specific network architectures with few or no 1x1 convolutional layers, for which standard binarization methods do not work well. Because 1x1 convolutions are common in the design of modern architectures (e.g. GoogleNet, ResNet, DenseNet), it is crucial to develop a method to binarize them effectively for BNNs to be more widely adopted. In this work, we propose an "Elastic-Link" (EL) module to enrich information flow within a BNN by adaptively adding real-valued input features to the subsequent convolutional output features. The proposed EL module is easily implemented and can be used in conjunction with other methods for BNNs. We demonstrate that adding EL to BNNs produces a significant improvement on the challenging large-scale ImageNet dataset. For example, we raise the top-1 accuracy of binarized ResNet26 from 57.9 aids convergence in the training of binarized MobileNet, for which a top-1 accuracy of 56.4 yields a new state-of-the-art result of 71.9

READ FULL TEXT
research
06/19/2019

Back to Simplicity: How to Train Accurate BNNs from Scratch?

Binary Neural Networks (BNNs) show promising progress in reducing comput...
research
09/11/2020

SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks

Binary Convolutional Neural Networks (BCNNs) can significantly improve t...
research
08/13/2023

Neural Networks at a Fraction with Pruned Quaternions

Contemporary state-of-the-art neural networks have increasingly large nu...
research
07/20/2018

Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy

Deep convolution neural network has achieved great success in many artif...
research
07/31/2019

EMPNet: Neural Localisation and Mapping using Embedded Memory Points

Continuously estimating an agent's state space and a representation of i...
research
07/25/2018

Crossbar-aware neural network pruning

Crossbar architecture based devices have been widely adopted in neural n...
research
06/24/2022

How to train accurate BNNs for embedded systems?

A key enabler of deploying convolutional neural networks on resource-con...

Please sign up or login with your details

Forgot password? Click here to reset