-
A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
The risks and perils of overfitting in machine learning are well known. ...
06/20/2018 ∙ by Amy Zhang, et al. ∙
0
∙
share
read it
-
Group Equivariant Deep Reinforcement Learning
In Reinforcement Learning (RL), Convolutional Neural Networks(CNNs) have...
07/01/2020 ∙ by Arnab Kumar Mondal, et al. ∙
0
∙
share
read it
-
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization
Recent results in Reinforcement Learning (RL) have shown that agents wit...
06/02/2019 ∙ by Xingyou Song, et al. ∙
0
∙
share
read it
-
Diagnosing Bottlenecks in Deep Q-learning Algorithms
Q-learning methods represent a commonly used class of algorithms in rein...
02/26/2019 ∙ by Justin Fu, et al. ∙
0
∙
share
read it
-
On overfitting and asymptotic bias in batch reinforcement learning with partial observability
This paper stands in the context of reinforcement learning with partial ...
09/22/2017 ∙ by Vincent Francois-Lavet, et al. ∙
0
∙
share
read it
-
The Principle of Unchanged Optimality in Reinforcement Learning Generalization
Several recent papers have examined generalization in reinforcement lear...
06/02/2019 ∙ by Alex Irpan, et al. ∙
2
∙
share
read it
-
Explore and Exploit with Heterotic Line Bundle Models
We use deep reinforcement learning to explore a class of heterotic SU(5)...
03/10/2020 ∙ by Magdalena Larfors, et al. ∙
0
∙
share
read it
A Study on Overfitting in Deep Reinforcement Learning
04/18/2018 ∙ by Chiyuan Zhang, et al. ∙
0
∙
share
Recent years have witnessed significant progresses in deep Reinforcement Learning (RL). Empowered with large scale neural networks, carefully designed architectures, novel training algorithms and massively parallel computing devices, researchers are able to attack many challenging RL problems. However, in machine learning, more training power comes with a potential risk of more overfitting. As deep RL techniques are being applied to critical problems such as healthcare and finance, it is important to understand the generalization behaviors of the trained agents. In this paper, we conduct a systematic study of standard RL agents and find that they could overfit in various ways. Moreover, overfitting could happen "robustly": commonly used techniques in RL that add stochasticity do not necessarily prevent or detect overfitting. In particular, the same agents and learning algorithms could have drastically different test performance, even when all of them achieve optimal rewards during training. The observations call for more principled and careful evaluation protocols in RL. We conclude with a general discussion on overfitting in RL and a study of the generalization behaviors from the perspective of inductive bias.
READ FULL TEXT
Comments
There are no comments yet.