An Algorithm for Routing Capsules in All Domains

11/02/2019 ∙ by Franz A. Heinsen, et al. ∙ 55

Building on recent work on capsule networks, we propose a new form of "routing by agreement" that activates output capsules in a layer as a function of their net benefit to use and net cost to ignore input capsules from earlier layers. As sample applications, we present two capsule networks that use our algorithm without change in different domains: vision and language. The first network achieves new state-of-the-art accuracy of 99.1 recognition task with fewer parameters and an order of magnitude less training than previous capsule models, and we find evidence that it learns to perform a form of "reverse graphics." The second network achieves new state-of-the-art accuracies on the root sentences of the Stanford Sentiment Treebank: 58.5 fine-grained and 95.6 frozen embeddings from a pretrained transformer as capsules. Both networks are trained with the same regime. Code is available at https://github.com/glassroom/heinsen_routing along with replication instructions.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 17

Code Repositories

heinsen_routing

Official implementation of "An Algorithm for Routing Capsules in All Domains" (Heinsen, 2019) in PyTorch.


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.