DeepAI AI Chat
Log In Sign Up

Kanerva++: extending The Kanerva Machine with differentiable, locally block allocated latent memory

by   Jason Ramapuram, et al.
Apple Inc.

Episodic and semantic memory are critical components of the human memory model. The theory of complementary learning systems (McClelland et al., 1995) suggests that the compressed representation produced by a serial event (episodic memory) is later restructured to build a more generalized form of reusable knowledge (semantic memory). In this work we develop a new principled Bayesian memory allocation scheme that bridges the gap between episodic and semantic memory via a hierarchical latent variable model. We take inspiration from traditional heap allocation and extend the idea of locally contiguous memory to the Kanerva Machine, enabling a novel differentiable block allocated latent memory. In contrast to the Kanerva Machine, we simplify the process of memory writing by treating it as a fully feed forward deterministic process, relying on the stochasticity of the read key distribution to disperse information within the memory. We demonstrate that this allocation scheme improves performance in memory conditional image generation, resulting in new state-of-the-art conditional likelihood values on binarized MNIST (<=41.58 nats/image) , binarized Omniglot (<=66.24 nats/image), as well as presenting competitive performance on CIFAR10, DMLab Mazes, Celeb-A and ImageNet32x32.


Conditional Variational Autoencoder for Neural Machine Translation

We explore the performance of latent variable models for conditional tex...

Learning to Learn Variational Semantic Memory

In this paper, we introduce variational semantic memory into meta-learni...

Guided Image Generation with Conditional Invertible Neural Networks

In this work, we address the task of natural image generation guided by ...

The Tensor Memory Hypothesis

We discuss memory models which are based on tensor decompositions using ...

Hierarchical Variational Memory for Few-shot Learning Across Domains

Neural memory enables fast adaptation to new tasks with just a few train...

Product Kanerva Machines: Factorized Bayesian Memory

An ideal cognitively-inspired memory system would compress and organize ...

The Kanerva Machine: A Generative Distributed Memory

We present an end-to-end trained memory system that quickly adapts to ne...