How do Offline Measures for Exploration in Reinforcement Learning behave?

10/29/2020
by   Jakob J. Hollenstein, et al.
0

Sufficient exploration is paramount for the success of a reinforcement learning agent. Yet, exploration is rarely assessed in an algorithm-independent way. We compare the behavior of three data-based, offline exploration metrics described in the literature on intuitive simple distributions and highlight problems to be aware of when using them. We propose a fourth metric,uniform relative entropy, and implement it using either a k-nearest-neighbor or a nearest-neighbor-ratio estimator, highlighting that the implementation choices have a profound impact on these measures.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

10/27/2021

Nearest neighbor process: weak convergence and non-asymptotic bound

An empirical measure that results from the nearest neighbors to a given ...
02/10/2021

Leveraging Reinforcement Learning for evaluating Robustness of KNN Search Algorithms

The problem of finding K-nearest neighbors in the given dataset for a gi...
02/03/2021

Nearest Neighbor-based Importance Weighting

Importance weighting is widely applicable in machine learning in general...
06/24/2019

An Empirical Comparison of FAISS and FENSHSES for Nearest Neighbor Search in Hamming Space

In this paper, we compare the performances of FAISS and FENSHSES on near...
06/20/2020

Coconut Palm: Static and Streaming Data Series Exploration Now in your Palm

Many modern applications produce massive streams of data series and main...
05/25/2019

Learning to Reason in Large Theories without Imitation

Automated theorem proving in large theories can be learned via reinforce...
05/18/2018

Graphon estimation via nearest neighbor algorithm and 2D fused lasso denoising

We propose a class of methods for graphon estimation based on exploiting...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.