REM: Routing Entropy Minimization for Capsule Networks

04/04/2022
by   Riccardo Renzulli, et al.
0

Capsule Networks ambition is to build an explainable and biologically-inspired neural network model. One of their main innovations relies on the routing mechanism which extracts a parse tree: its main purpose is to explicitly build relationships between capsules. However, their true potential in terms of explainability has not surfaced yet: these relationships are extremely heterogeneous and difficult to understand. This paper proposes REM, a technique which minimizes the entropy of the parse tree-like structure, improving its explainability. We accomplish this by driving the model parameters distribution towards low entropy configurations, using a pruning mechanism as a proxy. We also generate static parse trees with no performance loss, showing that, with REM, Capsule Networks build stronger relationships between capsules.

READ FULL TEXT

page 7

page 8

research
10/04/2020

Learning Compositional Structures for Deep Learning: Why Routing-by-agreement is Necessary

A formal description of the compositionality of neural networks is assoc...
research
06/12/2018

Capsule Routing for Sound Event Detection

The detection of acoustic scenes is a challenging problem in which envir...
research
04/01/2020

An Efficient Agreement Mechanism in CapsNets By Pairwise Product

Capsule networks (CapsNets) are capable of modeling visual hierarchical ...
research
01/05/2022

Quantum Capsule Networks

Capsule networks, which incorporate the paradigms of connectionism and s...
research
11/26/2019

TimeCaps: Capturing Time Series Data with Capsule Networks

Capsule networks excel in understanding spatial relationships in 2D data...
research
01/04/2023

Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption

Capsule neural networks replace simple, scalar-valued neurons with vecto...
research
10/15/2018

A Context-aware Capsule Network for Multi-label Classification

Recently proposed Capsule Network is a brain inspired architecture that ...

Please sign up or login with your details

Forgot password? Click here to reset