Efficient-CapsNet: Capsule Network with Self-Attention Routing

01/29/2021
by   Vittorio Mazzia, et al.
14

Deep convolutional neural networks, assisted by architectural design strategies, make extensive use of data augmentation techniques and layers with a high number of feature maps to embed object transformations. That is highly inefficient and for large datasets implies a massive redundancy of features detectors. Even though capsules networks are still in their infancy, they constitute a promising solution to extend current convolutional networks and endow artificial visual perception with a process to encode more efficiently all feature affine transformations. Indeed, a properly working capsule network should theoretically achieve higher results with a considerably lower number of parameters count due to intrinsic capability to generalize to novel viewpoints. Nevertheless, little attention has been given to this relevant aspect. In this paper, we investigate the efficiency of capsule networks and, pushing their capacity to the limits with an extreme architecture with barely 160K parameters, we prove that the proposed architecture is still able to achieve state-of-the-art results on three different datasets with only 2 original CapsNet parameters. Moreover, we replace dynamic routing with a novel non-iterative, highly parallelizable routing algorithm that can easily cope with a reduced number of capsules. Extensive experimentation with other capsule implementations has proved the effectiveness of our methodology and the capability of capsule networks to efficiently embed visual representations more prone to generalization.

READ FULL TEXT

page 8

page 10

page 11

research
08/27/2018

Generalized Capsule Networks with Trainable Routing Procedure

CapsNet (Capsule Network) was first proposed by capsule and later anothe...
research
09/19/2021

Capsule networks with non-iterative cluster routing

Capsule networks use routing algorithms to flow information between cons...
research
04/09/2019

Assessing Capsule Networks With Biased Data

Machine learning based methods achieves impressive results in object cla...
research
05/28/2018

Fast Dynamic Routing Based on Weighted Kernel Density Estimation

Capsules as well as dynamic routing between them are most recently propo...
research
07/09/2020

DECAPS: Detail-Oriented Capsule Networks

Capsule Networks (CapsNets) have demonstrated to be a promising alternat...
research
02/11/2019

Path Capsule Networks

Capsule network (CapsNet) was introduced as an enhancement over convolut...
research
06/13/2018

Group Equivariant Capsule Networks

We present group equivariant capsule networks, a framework to introduce ...

Please sign up or login with your details

Forgot password? Click here to reset