Log In Sign Up

Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Models

by   Jan Achterhold, et al.

In this paper, we learn dynamics models for parametrized families of dynamical systems with varying properties. The dynamics models are formulated as stochastic processes conditioned on a latent context variable which is inferred from observed transitions of the respective system. The probabilistic formulation allows us to compute an action sequence which, for a limited number of environment interactions, optimally explores the given system within the parametrized family. This is achieved by steering the system through transitions being most informative for the context variable. We demonstrate the effectiveness of our method for exploration on a non-linear toy-problem and two well-known reinforcement learning environments.


LEADS: Learning Dynamical Systems that Generalize Across Environments

When modeling dynamical systems from real-world data samples, the distri...

DQN with model-based exploration: efficient learning on environments with sparse rewards

We propose Deep Q-Networks (DQN) with model-based exploration, an algori...

The Challenges of Exploration for Offline Reinforcement Learning

Offline Reinforcement Learning (ORL) enablesus to separately study the t...

Variational Bridge Constructs for Grey Box Modelling with Gaussian Processes

This paper introduces a method for inference of heterogeneous dynamical ...

Task-Optimal Exploration in Linear Dynamical Systems

Exploration in unknown environments is a fundamental problem in reinforc...

Unsupervised Learning for Nonlinear PieceWise Smooth Hybrid Systems

This paper introduces a novel system identification and tracking method ...

A Note on Kernel Methods for Multiscale Systems with Critical Transitions

We study the maximum mean discrepancy (MMD) in the context of critical t...