A Theory of Abstraction in Reinforcement Learning

03/01/2022
by   David Abel, et al.
0

Reinforcement learning defines the problem facing agents that learn to make good decisions through action and observation alone. To be effective problem solvers, such agents must efficiently explore vast worlds, assign credit from delayed feedback, and generalize to new experiences, all while making use of limited data, computational resources, and perceptual bandwidth. Abstraction is essential to all of these endeavors. Through abstraction, agents can form concise models of their environment that support the many practices required of a rational, adaptive decision maker. In this dissertation, I present a theory of abstraction in reinforcement learning. I first offer three desiderata for functions that carry out the process of abstraction: they should 1) preserve representation of near-optimal behavior, 2) be learned and constructed efficiently, and 3) lower planning or learning time. I then present a suite of new algorithms and analysis that clarify how agents can learn to abstract according to these desiderata. Collectively, these results provide a partial path toward the discovery and use of abstraction that minimizes the complexity of effective reinforcement learning.

READ FULL TEXT
research
06/07/2022

Discrete State-Action Abstraction via the Successor Representation

When reinforcement learning is applied with sparse rewards, agents must ...
research
03/13/2019

Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems

Real-world congestion problems (e.g. traffic congestion) are typically v...
research
11/13/2019

Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learning

We present an algorithm, HOMER, for exploration and reinforcement learni...
research
09/23/2020

The Agent Web Model – Modelling web hacking for reinforcement learning

Website hacking is a frequent attack type used by malicious actors to ob...
research
07/29/2019

Goal-Driven Sequential Data Abstraction

Automatic data abstraction is an important capability for both benchmark...
research
11/18/2017

Learning to select computations

Efficient use of limited computational resources is essential to intelli...
research
10/13/2022

A Direct Approximation of AIXI Using Logical State Abstractions

We propose a practical integration of logical state abstraction with AIX...

Please sign up or login with your details

Forgot password? Click here to reset