Network Sketching: Exploiting Binary Structure in Deep CNNs

06/07/2017
by   Yiwen Guo, et al.
0

Convolutional neural networks (CNNs) with deep architectures have substantially advanced the state-of-the-art in computer vision tasks. However, deep networks are typically resource-intensive and thus difficult to be deployed on mobile devices. Recently, CNNs with binary weights have shown compelling efficiency to the community, whereas the accuracy of such models is usually unsatisfactory in practice. In this paper, we introduce network sketching as a novel technique of pursuing binary-weight CNNs, targeting at more faithful inference and better trade-off for practical applications. Our basic idea is to exploit binary structure directly in pre-trained filter banks and produce binary-weight models via tensor expansion. The whole process can be treated as a coarse-to-fine model approximation, akin to the pencil drawing steps of outlining and shading. To further speedup the generated models, namely the sketches, we also propose an associative implementation of binary tensor convolutions. Experimental results demonstrate that a proper sketch of AlexNet (or ResNet) outperforms the existing binary-weight models by large margins on the ImageNet large scale classification task, while the committed memory for network parameters only exceeds a little.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2018

From Hashing to CNNs: Training BinaryWeight Networks via Hashing

Deep convolutional neural networks (CNNs) have shown appealing performan...
research
08/08/2020

Towards Lossless Binary Convolutional Neural Networks Using Piecewise Approximation

Binary Convolutional Neural Networks (CNNs) can significantly reduce the...
research
09/25/2019

Accurate and Compact Convolutional Neural Networks with Trained Binarization

Although convolutional neural networks (CNNs) are now widely used in var...
research
03/16/2016

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

We propose two efficient approximations to standard convolutional neural...
research
05/26/2020

Learning to map between ferns with differentiable binary embedding networks

Current deep learning methods are based on the repeated, expensive appli...
research
03/23/2022

Binary Morphological Neural Network

In the last ten years, Convolutional Neural Networks (CNNs) have formed ...
research
10/03/2020

Nonconvex Regularization for Network Slimming:Compressing CNNs Even More

In the last decade, convolutional neural networks (CNNs) have evolved to...

Please sign up or login with your details

Forgot password? Click here to reset