Bad-Policy Density: A Measure of Reinforcement Learning Hardness

10/07/2021
by   David Abel, et al.
7

Reinforcement learning is hard in general. Yet, in many specific environments, learning is easy. What makes learning easy in one environment, but difficult in another? We address this question by proposing a simple measure of reinforcement-learning hardness called the bad-policy density. This quantity measures the fraction of the deterministic stationary policy space that is below a desired threshold in value. We prove that this simple quantity has many properties one would expect of a measure of learning hardness. Further, we prove it is NP-hard to compute the measure in general, but there are paths to polynomial-time approximation. We conclude by summarizing potential directions and uses for this measure.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/07/2019

A hardness of approximation result in metric geometry

We show that it is NP-hard to approximate the hyperspherical radius of a...
research
10/24/2022

Hardness in Markov Decision Processes: Theory and Practice

Meticulously analysing the empirical strengths and weaknesses of reinfor...
research
07/06/2017

Hardness of learning noisy halfspaces using polynomial thresholds

We prove the hardness of weakly learning halfspaces in the presence of a...
research
03/14/2018

Removing Skill Bias from Gaming Statistics

"The chance to win given a certain move" is an easily obtainable quantit...
research
11/15/2022

Shellability is hard even for balls

The main goal of this paper is to show that shellability is NP-hard for ...
research
11/14/2019

Hardness of Learning DNFs using Halfspaces

The problem of learning t-term DNF formulas (for t = O(1)) has been stud...
research
09/05/2018

Complexity of 2D bootstrap percolation difficulty: Algorithm and NP-hardness

Bootstrap percolation is a class of cellular automata with random initia...

Please sign up or login with your details

Forgot password? Click here to reset