How do Offline Measures for Exploration in Reinforcement Learning behave?

10/29/2020
by   Jakob J. Hollenstein, et al.
0

Sufficient exploration is paramount for the success of a reinforcement learning agent. Yet, exploration is rarely assessed in an algorithm-independent way. We compare the behavior of three data-based, offline exploration metrics described in the literature on intuitive simple distributions and highlight problems to be aware of when using them. We propose a fourth metric,uniform relative entropy, and implement it using either a k-nearest-neighbor or a nearest-neighbor-ratio estimator, highlighting that the implementation choices have a profound impact on these measures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2021

Nearest neighbor process: weak convergence and non-asymptotic bound

An empirical measure that results from the nearest neighbors to a given ...
research
03/08/2022

Rényi State Entropy for Exploration Acceleration in Reinforcement Learning

One of the most critical challenges in deep reinforcement learning is to...
research
02/10/2021

Leveraging Reinforcement Learning for evaluating Robustness of KNN Search Algorithms

The problem of finding K-nearest neighbors in the given dataset for a gi...
research
02/03/2021

Nearest Neighbor-based Importance Weighting

Importance weighting is widely applicable in machine learning in general...
research
05/25/2019

Learning to Reason in Large Theories without Imitation

Automated theorem proving in large theories can be learned via reinforce...
research
06/20/2020

Coconut Palm: Static and Streaming Data Series Exploration Now in your Palm

Many modern applications produce massive streams of data series and main...
research
12/12/2018

Prediction of Success or Failure for Final Examination using Nearest Neighbor Method to the Trend of Weekly Online Testing

Using the trends of estimated abilities in terms of item response theory...

Please sign up or login with your details

Forgot password? Click here to reset