DeepAI AI Chat
Log In Sign Up

Capacity Analysis of Vector Symbolic Architectures

01/24/2023
by   Kenneth L. Clarkson, et al.
0

Hyperdimensional computing (HDC) is a biologically-inspired framework which represents symbols with high-dimensional vectors, and uses vector operations to manipulate them. The ensemble of a particular vector space and a prescribed set of vector operations (including one addition-like for "bundling" and one outer-product-like for "binding") form a *vector symbolic architecture* (VSA). While VSAs have been employed in numerous applications and have been studied empirically, many theoretical questions about VSAs remain open. We analyze the *representation capacities* of four common VSAs: MAP-I, MAP-B, and two VSAs based on sparse binary vectors. "Representation capacity' here refers to bounds on the dimensions of the VSA vectors required to perform certain symbolic tasks, such as testing for set membership i ∈ S and estimating set intersection sizes |X ∩ Y| for two sets of symbols X and Y, to a given degree of accuracy. We also analyze the ability of a novel variant of a Hopfield network (a simple model of associative memory) to perform some of the same tasks that are typically asked of VSAs. In addition to providing new bounds on VSA capacities, our analyses establish and leverage connections between VSAs, "sketching" (dimensionality reduction) algorithms, and Bloom filters.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/31/2020

A comparison of Vector Symbolic Architectures

Vector Symbolic Architectures (VSAs) combine a high-dimensional vector s...
07/18/2022

Residual and Attentional Architectures for Vector-Symbols

Vector-symbolic architectures (VSAs) provide methods for computing which...
09/14/2020

Variable Binding for Sparse Distributed Representations: Theory and Applications

Symbolic reasoning and neural networks are often considered incompatible...
09/30/2020

Analyzing the Capacity of Distributed Vector Representations to Encode Spatial Information

Vector Symbolic Architectures belong to a family of related cognitive mo...
09/24/2018

Representing Sets as Summed Semantic Vectors

Representing meaning in the form of high dimensional vectors is a common...
01/19/2021

Hyperdimensional computing as a framework for systematic aggregation of image descriptors

Image and video descriptors are an omnipresent tool in computer vision a...
09/05/2021

Learning with Holographic Reduced Representations

Holographic Reduced Representations (HRR) are a method for performing sy...