DeepAI AI Chat
Log In Sign Up

Learning Plannable Representations with Causal InfoGAN

07/24/2018
by   Thanard Kurutach, et al.
berkeley college
8

In recent years, deep generative models have been shown to 'imagine' convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data. In this work, we ask how to imagine goal-directed visual plans -- a plausible sequence of observations that transition a dynamical system from its current configuration to a desired goal state, which can later be used as a reference trajectory for control. We focus on systems with high-dimensional observations, such as images, and propose an approach that naturally combines representation learning and planning. Our framework learns a generative model of sequential observations, where the generative process is induced by a transition in a low-dimensional planning model, and an additional noise. By maximizing the mutual information between the generated observations and the transition in the planning model, we obtain a low-dimensional representation that best explains the causal nature of the data. We structure the planning model to be compatible with efficient planning algorithms, and we propose several such models based on either discrete or continuous states. Finally, to generate a visual plan, we project the current and goal observations onto their respective states in the planning model, plan a trajectory, and then use the generative model to transform the trajectory to a sequence of observations. We demonstrate our method on imagining plausible visual plans of rope manipulation.

READ FULL TEXT
05/11/2019

Learning Robotic Manipulation through Visual Planning and Acting

Planning for robotic manipulation requires reasoning about the changes a...
02/27/2020

Hallucinative Topological Memory for Zero-Shot Visual Planning

In visual planning (VP), an agent learns to plan goal-directed behavior ...
07/05/2018

Adaptive Path-Integral Approach to Representation Learning and Planning for Dynamical Systems

We present a representation learning algorithm that learns a low-dimensi...
07/05/2018

Adaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems

We present a representation learning algorithm that learns a low-dimensi...
03/03/2021

Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmap

We present a framework for visual action planning of complex manipulatio...
11/25/2022

Learning Visual Planning Models from Partially Observed Images

There has been increasing attention on planning model learning in classi...

Code Repositories

curated-list-of-interesting-topics-in-ml-and-maths

My curated list of articles and topics I find interesting.


view repo

chainer-causal-info-gan

Reproduction codes of Causal Info GAN with chainer


view repo

PhysicallyVerifiableCausalInfoGan

An extension to causal infoGan


view repo