Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control

06/15/2020
by   Rika Antonova, et al.
7

We address the problem of learning reusable state representations from streaming high-dimensional observations. This is important for areas like Reinforcement Learning (RL), which yields non-stationary data distributions during training. We make two key contributions. First, we propose an evaluation suite that measures alignment between latent and true low-dimensional states. We benchmark several widely used unsupervised learning approaches. This uncovers the strengths and limitations of existing approaches that impose additional constraints/objectives on the latent space. Our second contribution is a unifying mathematical formulation for learning latent relations. We learn analytic relations on source domains, then use these relations to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. We present mathematical properties, concrete algorithms for implementation and experimental validation of successful learning and transfer of latent relations.

READ FULL TEXT

page 2

page 7

page 11

06/06/2019

DeepMDP: Learning Continuous Latent Space Models for Representation Learning

Many reinforcement learning (RL) tasks provide the agent with high-dimen...
06/03/2022

Reinforcement Learning with Neural Radiance Fields

It is a long-standing problem to find effective representations for trai...
07/10/2018

Representation Learning with Contrastive Predictive Coding

While supervised learning has enabled great progress in many application...
06/24/2020

Control-Aware Representations for Model-based Reinforcement Learning

A major challenge in modern reinforcement learning (RL) is efficient con...
07/10/2021

LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Iterative Tasks

Reinforcement learning (RL) algorithms have shown impressive success in ...
05/08/2021

Adaptive Latent Space Tuning for Non-Stationary Distributions

Powerful deep learning tools, such as convolutional neural networks (CNN...
06/21/2019

Connectivity-Optimized Representation Learning via Persistent Homology

We study the problem of learning representations with controllable conne...

Code Repositories

bulb

Benchmarking Unsupervised Learning with pyBullet


view repo

Please sign up or login with your details

Forgot password? Click here to reset