Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning

by   Bram Grooten, et al.

Tomorrow's robots will need to distinguish useful information from noise when performing different tasks. A household robot for instance may continuously receive a plethora of information about the home, but needs to focus on just a small subset to successfully execute its current chore. Filtering distracting inputs that contain irrelevant data has received little attention in the reinforcement learning literature. To start resolving this, we formulate a problem setting in reinforcement learning called the extremely noisy environment (ENE), where up to 99% of the input features are pure noise. Agents need to detect which features provide task-relevant information about the state of the environment. Consequently, we propose a new method termed Automatic Noise Filtering (ANF), which uses the principles of dynamic sparse training in synergy with various deep reinforcement learning algorithms. The sparse input layer learns to focus its connectivity on task-relevant features, such that ANF-SAC and ANF-TD3 outperform standard SAC and TD3 by a large margin, while using up to 95% fewer weights. Furthermore, we devise a transfer learning setting for ENEs, by permuting all features of the environment after 1M timesteps to simulate the fact that other information sources can become relevant as the world evolves. Again, ANF surpasses the baselines in final performance and sample complexity. Our code is available at


page 6

page 13

page 16


Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Arm

As technology progresses, industrial and scientific robots are increasin...

Noise, overestimation and exploration in Deep Reinforcement Learning

We will discuss some statistical noise related phenomena, that were inve...

MicroRacer: a didactic environment for Deep Reinforcement Learning

MicroRacer is a simple, open source environment inspired by car racing e...

Generalization in Transfer Learning

Agents trained with deep reinforcement learning algorithms are capable o...

ToriLLE: Learning Environment for Hand-to-Hand Combat

We present Toribash Learning Environment (ToriLLE), an interface with vi...

Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning

Reinforcement Learning enables to train an agent via interaction with th...

Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order

In this work, we analyze the performance of general deep reinforcement l...

Please sign up or login with your details

Forgot password? Click here to reset