Working Memory for Online Memory Binding Tasks: A Hybrid Model

Working Memory is the brain module that holds and manipulates information online. In this work, we design a hybrid model in which a simple feed-forward network is coupled to a balanced random network via a read-write vector called the interface vector. First, we consider some simple memory binding tasks in which the output is set to be a copy of the given input and a selective sequence of previous inputs online. Next, we design a more complex binding task based on a cue that encodes binding relations. The important result is that our dual-component model of working memory shows good performance with learning restricted to the feed-forward component only. Here we take advantage of the random network property without learning. To our knowledge, this is the first time that random networks as a flexible memory is shown to play an important role in online binding tasks. We may interpret our results as a candidate model of working memory in which the feed-forward network learns to interact with the temporary storage random network as an attentional-controlling executive system.

READ FULL TEXT

page 4

page 6

page 7

page 12

page 13

page 14

page 18

research
12/29/2015

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

We propose a simplified model of attention which is applicable to feed-f...
research
02/07/2019

Hybrid Models with Deep and Invertible Features

We propose a neural hybrid model consisting of a linear model defined on...
research
08/19/2022

Kernel Memory Networks: A Unifying Framework for Memory Modeling

We consider the problem of training a neural network to store a set of p...
research
08/01/2017

Natural Language Processing with Small Feed-Forward Networks

We show that small and shallow feed-forward neural networks can achieve ...
research
01/15/2022

Kformer: Knowledge Injection in Transformer Feed-Forward Layers

Knowledge-Enhanced Model have developed a diverse set of techniques for ...
research
01/17/2020

A Robust Model of Gated Working Memory

Gated working memory is defined as the capacity of holding arbitrary inf...
research
04/01/2020

Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning

The Obstacle Tower Challenge is the task to master a procedurally genera...

Please sign up or login with your details

Forgot password? Click here to reset