BAM: Bayes with Adaptive Memory

02/04/2022
by   Josue Nassar, et al.
11

Online learning via Bayes' theorem allows new data to be continuously integrated into an agent's current beliefs. However, a naive application of Bayesian methods in non stationary environments leads to slow adaptation and results in state estimates that may converge confidently to the wrong parameter value. A common solution when learning in changing environments is to discard/downweight past data; however, this simple mechanism of "forgetting" fails to account for the fact that many real-world environments involve revisiting similar states. We propose a new framework, Bayes with Adaptive Memory (BAM), that takes advantage of past experience by allowing the agent to choose which past observations to remember and which to forget. We demonstrate that BAM generalizes many popular Bayesian update rules for non-stationary environments. Through a variety of experiments, we demonstrate the ability of BAM to continuously adapt in an ever-changing world.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/23/2022

Learning Fast and Slow for Online Time Series Forecasting

The fast adaptation capability of deep neural networks in non-stationary...
research
07/31/2017

Taming Non-stationary Bandits: A Bayesian Approach

We consider the multi armed bandit problem in non-stationary environment...
research
02/20/2023

Adaptive Sparse Gaussian Process

Adaptive learning is necessary for non-stationary environments where the...
research
12/07/2020

Reset-Free Lifelong Learning with Skill-Space Planning

The objective of lifelong reinforcement learning (RL) is to optimize age...
research
07/05/2019

An Approximate Bayesian Approach to Surprise-Based Learning

Surprise-based learning allows agents to adapt quickly in non-stationary...
research
09/18/2020

HTMRL: Biologically Plausible Reinforcement Learning with Hierarchical Temporal Memory

Building Reinforcement Learning (RL) algorithms which are able to adapt ...
research
12/16/2020

Lévy walks derived from a Bayesian decision-making model in non-stationary environments

Lévy walks are found in the migratory behaviour patterns of various orga...

Please sign up or login with your details

Forgot password? Click here to reset