Optimal quadratic binding for relational reasoning in vector symbolic neural architectures

04/14/2022
by   Naoki Hiratani, et al.
0

Binding operation is fundamental to many cognitive processes, such as cognitive map formation, relational reasoning, and language comprehension. In these processes, two different modalities, such as location and objects, events and their contextual cues, and words and their roles, need to be bound together, but little is known about the underlying neural mechanisms. Previous works introduced a binding model based on quadratic functions of bound pairs, followed by vector summation of multiple pairs. Based on this framework, we address following questions: Which classes of quadratic matrices are optimal for decoding relational structures? And what is the resultant accuracy? We introduce a new class of binding matrices based on a matrix representation of octonion algebra, an eight-dimensional extension of complex numbers. We show that these matrices enable a more accurate unbinding than previously known methods when a small number of pairs are present. Moreover, numerical optimization of a binding operator converges to this octonion binding. We also show that when there are a large number of bound pairs, however, a random quadratic binding performs as well as the octonion and previously-proposed binding methods. This study thus provides new insight into potential neural mechanisms of binding operations in the brain.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/01/2018

SARN: Relational Reasoning through Sequential Attention

This paper proposes an attention module augmented relational network cal...
research
05/23/2018

Working Memory Networks: Augmenting Memory Networks with a Relational Reasoning Module

During the last years, there has been a lot of interest in achieving som...
research
10/20/2022

CLT for random quadratic forms based on sample means and sample covariance matrices

In this paper, we use the dimensional reduction technique to study the c...
research
06/09/2019

Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound

We improve the over-parametrization size over two beautiful results [Li ...
research
06/18/2019

Expressing Visual Relationships via Language

Describing images with text is a fundamental problem in vision-language ...
research
06/10/2021

Temporal and Object Quantification Networks

We present Temporal and Object Quantification Networks (TOQ-Nets), a new...

Please sign up or login with your details

Forgot password? Click here to reset