Unsupervised Real-Time Control through Variational Empowerment

by   Maximilian Karl, et al.

We introduce a methodology for efficiently computing a lower bound to empowerment, allowing it to be used as an unsupervised cost function for policy learning in real-time control. Empowerment, being the channel capacity between actions and states, maximises the influence of an agent on its near future. It has been shown to be a good model of biological behaviour in the absence of an extrinsic goal. But empowerment is also prohibitively hard to compute, especially in nonlinear continuous spaces. We introduce an efficient, amortised method for learning empowerment-maximising policies. We demonstrate that our algorithm can reliably handle continuous dynamical systems using system dynamics learned from raw data. The resulting policies consistently drive the agents into states where they can use their full potential.



page 6

page 8


Hamilton-Jacobi-Bellman Equations for Q-Learning in Continuous Time

In this paper, we introduce Hamilton-Jacobi-Bellman (HJB) equations for ...

A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight

Quadrotors are highly nonlinear dynamical systems that require carefully...

On Applications of Bootstrap in Continuous Space Reinforcement Learning

In decision making problems for continuous state and action spaces, line...

Structured Neural Network Dynamics for Model-based Control

We present a structured neural network architecture that is inspired by ...

Re-purposing Compact Neuronal Circuit Policies to Govern Reinforcement Learning Tasks

We propose an effective method for creating interpretable control agents...

Deep Reinforcement Learning for Event-Driven Multi-Agent Decision Processes

The incorporation of macro-actions (temporally extended actions) into mu...

Certification of Iterative Predictions in Bayesian Neural Networks

We consider the problem of computing reach-avoid probabilities for itera...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.