Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous Control

by   Rika Antonova, et al.

We address the problem of learning reusable state representations from streaming high-dimensional observations. This is important for areas like Reinforcement Learning (RL), which yields non-stationary data distributions during training. We make two key contributions. First, we propose an evaluation suite that measures alignment between latent and true low-dimensional states. We benchmark several widely used unsupervised learning approaches. This uncovers the strengths and limitations of existing approaches that impose additional constraints/objectives on the latent space. Our second contribution is a unifying mathematical formulation for learning latent relations. We learn analytic relations on source domains, then use these relations to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. We present mathematical properties, concrete algorithms for implementation and experimental validation of successful learning and transfer of latent relations.


page 2

page 7

page 11


DeepMDP: Learning Continuous Latent Space Models for Representation Learning

Many reinforcement learning (RL) tasks provide the agent with high-dimen...

Reinforcement Learning with Neural Radiance Fields

It is a long-standing problem to find effective representations for trai...

Representation Learning with Contrastive Predictive Coding

While supervised learning has enabled great progress in many application...

Control-Aware Representations for Model-based Reinforcement Learning

A major challenge in modern reinforcement learning (RL) is efficient con...

LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Iterative Tasks

Reinforcement learning (RL) algorithms have shown impressive success in ...

Adaptive Latent Space Tuning for Non-Stationary Distributions

Powerful deep learning tools, such as convolutional neural networks (CNN...

Connectivity-Optimized Representation Learning via Persistent Homology

We study the problem of learning representations with controllable conne...

Code Repositories


Benchmarking Unsupervised Learning with pyBullet

view repo

Please sign up or login with your details

Forgot password? Click here to reset