Local Binary Convolutional Neural Networks

08/22/2016
by   Felix Juefei-Xu, et al.
0

We propose local binary convolution (LBC), an efficient alternative to convolutional layers in standard convolutional neural networks (CNN). The design principles of LBC are motivated by local binary patterns (LBP). The LBC layer comprises of a set of fixed sparse pre-defined binary convolutional filters that are not updated during the training process, a non-linear activation function and a set of learnable linear weights. The linear weights combine the activated filter responses to approximate the corresponding activated filter responses of a standard convolutional layer. The LBC layer affords significant parameter savings, 9x to 169x in the number of learnable parameters compared to a standard convolutional layer. Furthermore, the sparse and binary nature of the weights also results in up to 9x to 169x savings in model size compared to a standard convolutional layer. We demonstrate both theoretically and experimentally that our local binary convolution layer is a good approximation of a standard convolutional layer. Empirically, CNNs with LBC layers, called local binary convolutional neural networks (LBCNN), achieves performance parity with regular CNNs on a range of visual datasets (MNIST, SVHN, CIFAR-10, and ImageNet) while enjoying significant computational savings.

READ FULL TEXT

page 3

page 4

research
06/05/2018

Perturbative Neural Networks

Convolutional neural networks are witnessing wide adoption in computer v...
research
09/14/2016

Understanding Convolutional Neural Networks with A Mathematical Model

This work attempts to address two fundamental questions about the struct...
research
12/09/2019

An Empirical Study on Position of the Batch Normalization Layer in Convolutional Neural Networks

In this paper, we have studied how the training of the convolutional neu...
research
10/08/2018

A look at the topology of convolutional neural networks

Convolutional neural networks (CNN's) are powerful and widely used tools...
research
06/19/2020

From Discrete to Continuous Convolution Layers

A basic operation in Convolutional Neural Networks (CNNs) is spatial res...
research
06/29/2020

Explainable 3D Convolutional Neural Networks by Learning Temporal Transformations

In this paper we introduce the temporally factorized 3D convolution (3TC...
research
11/28/2021

Implicit Equivariance in Convolutional Networks

Convolutional Neural Networks(CNN) are inherently equivariant under tran...

Please sign up or login with your details

Forgot password? Click here to reset