DeepAI
Log In Sign Up

Residual and Attentional Architectures for Vector-Symbols

Vector-symbolic architectures (VSAs) provide methods for computing which are highly flexible and carry unique advantages. Concepts in VSAs are represented by 'symbols,' long vectors of values which utilize properties of high-dimensional spaces to represent and manipulate information. In this new work, we combine efficiency of the operations provided within the framework of the Fourier Holographic Reduced Representation (FHRR) VSA with the power of deep networks to construct novel VSA based residual and attention-based neural network architectures. Using an attentional FHRR architecture, we demonstrate that the same network architecture can address problems from different domains (image classification and molecular toxicity prediction) by encoding different information into the network's inputs, similar to the Perceiver model. This demonstrates a novel application of VSAs and a potential path to implementing state-of-the-art neural models on neuromorphic hardware.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/07/2018

Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

Over the past few years, Spiking Neural Networks (SNNs) have become popu...
06/09/2021

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

This article reviews recent progress in the development of the computing...
09/08/2021

Computing on Functions Using Randomized Vector Representations

Vector space models for symbolic processing that encode symbols by rando...
11/01/2020

RANC: Reconfigurable Architecture for Neuromorphic Computing

Neuromorphic architectures have been introduced as platforms for energy ...
10/15/2021

HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

Motivated by recent innovations in biologically-inspired neuromorphic ha...
03/02/2022

Integer Factorization with Compositional Distributed Representations

In this paper, we present an approach to integer factorization using dis...
04/25/2021

Vector Neurons: A General Framework for SO(3)-Equivariant Networks

Invariance and equivariance to the rotation group have been widely discu...