Learning the Arrow of Time

07/02/2019
by   Nasim Rahaman, et al.
9

We humans seem to have an innate understanding of the asymmetric progression of time, which we use to efficiently and safely perceive and manipulate our environment. Drawing inspiration from that, we address the problem of learning an arrow of time in a Markov (Decision) Process. We illustrate how a learned arrow of time can capture meaningful information about the environment, which in turn can be used to measure reachability, detect side-effects and to obtain an intrinsic reward signal. We show empirical results on a selection of discrete and continuous environments, and demonstrate for a class of stochastic processes that the learned arrow of time agrees reasonably well with a known notion of an arrow of time given by the celebrated Jordan-Kinderlehrer-Otto result.

READ FULL TEXT

page 8

page 17

page 18

page 19

research
04/14/2010

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization

We study the convergence of Markov Decision Processes made of a large nu...
research
05/25/2023

Markov Decision Process with an External Temporal Process

Most reinforcement learning algorithms treat the context under which the...
research
12/17/2021

Distillation of RL Policies with Formal Guarantees via Variational Abstraction of Markov Decision Processes (Technical Report)

We consider the challenge of policy simplification and verification in t...
research
02/25/2022

Reachability analysis in stochastic directed graphs by reinforcement learning

We characterize the reachability probabilities in stochastic directed gr...
research
09/30/2022

Robust Q-learning Algorithm for Markov Decision Processes under Wasserstein Uncertainty

We present a novel Q-learning algorithm to solve distributionally robust...
research
11/22/2017

Budget Allocation in Binary Opinion Dynamics

In this article we study the allocation of a budget to promote an opinio...

Please sign up or login with your details

Forgot password? Click here to reset