Training Deep Capsule Networks

12/23/2018
by   David Peer, et al.
0

The capsules of Capsule Networks are collections of neurons that represent an object or part of an object in a parse tree. The output vector of a capsule encodes the so called instantiation parameters of this object (e.g. position, size, or orientation). The routing-by-agreement algorithm routes output vectors from lower level capsules to upper level capsules. This iterative algorithm selects the most appropriate parent capsule so that the active capsules in the network represent nodes in a parse tree. This parse tree represents the hierarchical composition of objects out of smaller and smaller components. In this paper, we will show experimentally that the routing-by-agreement algorithm does not ensure the emergence of a parse tree in the network. To ensure that all active capsules form a parse tree, we introduce a new routing algorithm called dynamic deep routing. We show that this routing algorithm allows the training of deeper capsule networks and is also more robust to white box adversarial attacks than the original routing algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/26/2017

Dynamic Routing Between Capsules

A capsule is a group of neurons whose activity vector represents the ins...
research
05/21/2019

Limitations of routing-by-agreement based capsule networks

Classical neural networks add a bias term to the sum of all weighted inp...
research
05/27/2019

Capsule Routing via Variational Bayes

Capsule Networks are a recently proposed alternative for constructing Ne...
research
10/04/2020

Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary

A formal description of the compositionality of neural networks is assoc...
research
07/26/2019

Training capsules as a routing-weighted product of expert neurons

Capsules are the multidimensional analogue to scalar neurons in neural n...
research
08/04/2019

Building Deep, Equivariant Capsule Networks

Capsule networks are constrained by their, relative, inability to deeper...
research
06/06/2019

Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications

Obstacles hindering the development of capsule networks for challenging ...

Please sign up or login with your details

Forgot password? Click here to reset