Unsupervised Real-Time Control through Variational Empowerment

10/13/2017
by   Maximilian Karl, et al.
0

We introduce a methodology for efficiently computing a lower bound to empowerment, allowing it to be used as an unsupervised cost function for policy learning in real-time control. Empowerment, being the channel capacity between actions and states, maximises the influence of an agent on its near future. It has been shown to be a good model of biological behaviour in the absence of an extrinsic goal. But empowerment is also prohibitively hard to compute, especially in nonlinear continuous spaces. We introduce an efficient, amortised method for learning empowerment-maximising policies. We demonstrate that our algorithm can reliably handle continuous dynamical systems using system dynamics learned from raw data. The resulting policies consistently drive the agents into states where they can use their full potential.

READ FULL TEXT

Authors

page 6

page 8

12/23/2019

Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time

In this paper, we introduce Hamilton-Jacobi-Bellman (HJB) equations for ...
02/22/2022

A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight

Quadrotors are highly nonlinear dynamical systems that require carefully...
03/14/2019

On Applications of Bootstrap in Continuous Space Reinforcement Learning

In decision making problems for continuous state and action spaces, line...
08/03/2018

Structured Neural Network Dynamics for Model-based Control

We present a structured neural network architecture that is inspired by ...
09/11/2018

Re-purposing Compact Neuronal Circuit Policies to Govern Reinforcement Learning Tasks

We propose an effective method for creating interpretable control agents...
09/19/2017

Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes

The incorporation of macro-actions (temporally extended actions) into mu...
05/21/2021

Certification of Iterative Predictions in Bayesian Neural Networks

We consider the problem of computing reach-avoid probabilities for itera...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.