Towards a practical measure of interference for reinforcement learning

07/07/2020
by   Vincent Liu, et al.
28

Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it. But, before we overcome interference we must understand it better. In this work, we provide a definition of interference for control in reinforcement learning. We systematically evaluate our new measures, by assessing correlation with several measures of learning performance, including stability, sample efficiency, and online and offline control performance across a variety of learning architectures. Our new interference measure allows us to ask novel scientific questions about commonly used deep learning architectures. In particular we show that target network frequency is a dominating factor for interference, and that updates on the last layer result in significantly higher interference than updates internal to the network. This new measure can be expensive to compute; we conclude with motivation for an efficient proxy measure and empirically demonstrate it is correlated with our definition of interference.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/10/2023

Measuring and Mitigating Interference in Reinforcement Learning

Catastrophic interference is common in many network-based learning syste...
research
10/29/2019

Overcoming Catastrophic Interference in Online Reinforcement Learning with Dynamic Self-Organizing Maps

Using neural networks in the reinforcement learning (RL) framework has a...
research
02/28/2020

On Catastrophic Interference in Atari 2600 Games

Model-free deep reinforcement learning algorithms are troubled with poor...
research
08/18/2021

A good body is all you need: avoiding catastrophic interference via agent architecture search

In robotics, catastrophic interference continues to restrain policy trai...
research
03/16/2020

Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks

Reinforcement learning systems require good representations to work well...
research
06/16/2018

DynMat, a network that can learn after learning

To survive in the dynamically-evolving world, we accumulate knowledge an...
research
06/25/2019

On the definition of likelihood function

We discuss a general definition of likelihood function in terms of Radon...

Please sign up or login with your details

Forgot password? Click here to reset