Online abstraction with MDP homomorphisms for Deep Learning

11/30/2018
by   Ondrej Biza, et al.
0

Abstraction of Markov Decision Processes is a useful tool for solving complex problems, as it can ignore unimportant aspects of an environment, simplifying the process of learning an optimal policy. In this paper, we propose a new algorithm for finding abstract MDPs in environments with continuous state spaces. It is based on MDP homomorphisms, a structure-preserving mapping between MDPs. We demonstrate our algorithm's ability to learns abstractions from collected experience and show how to reuse the abstractions to guide exploration in new tasks the agent encounters. Our novel task transfer method beats a baseline based on a deep Q-network.

READ FULL TEXT
research
07/20/2021

Similarity metrics for Different Market Scenarios in Abides

Markov Decision Processes (MDPs) are an effective way to formally descri...
research
05/28/2019

Planning with State Abstractions for Non-Markovian Task Specifications

Often times, we specify tasks for a robot using temporal language that c...
research
07/04/2012

Counterexample-guided Planning

Planning in adversarial and uncertain environments can be modeled as the...
research
11/10/2022

Switching Attention in Time-Varying Environments via Bayesian Inference of Abstractions

Motivated by the goal of endowing robots with a means for focusing atten...
research
06/13/2012

Speeding Up Planning in Markov Decision Processes via Automatically Constructed Abstractions

In this paper, we consider planning in stochastic shortest path (SSP) pr...
research
12/18/2021

Exploiting Expert-guided Symmetry Detection in Markov Decision Processes

Offline estimation of the dynamical model of a Markov Decision Process (...
research
10/08/2020

Adaptive Shielding under Uncertainty

This paper targets control problems that exhibit specific safety and per...

Please sign up or login with your details

Forgot password? Click here to reset