DeepAI
Log In Sign Up

Learning Latent Dynamics for Planning from Pixels

11/12/2018
by   Danijar Hafner, et al.
6

Planning has been very successful for control tasks with known environment dynamics. To leverage planning in unknown environments, the agent needs to learn the dynamics from interactions with the world. However, learning dynamics models that are accurate enough for planning has been a long-standing challenge, especially in image-based domains. We propose the Deep Planning Network (PlaNet), a purely model-based agent that learns the environment dynamics from pixels and chooses actions through online planning in latent space. To achieve high performance, the dynamics model must accurately predict the rewards ahead for multiple time steps. We approach this problem using a latent dynamics model with both deterministic and stochastic transition function and a generalized variational inference objective that we name latent overshooting. Using only pixel observations, our agent solves continuous control tasks with contact dynamics, partial observability, and sparse rewards. PlaNet uses significantly fewer episodes and reaches final performance close to and sometimes higher than top model-free algorithms.

READ FULL TEXT

page 2

page 15

page 17

05/07/2020

Planning from Images with Deep Latent Gaussian Process Dynamics

Planning is a powerful approach to control problems with known environme...
12/04/2020

Planning from Pixels using Inverse Dynamics Models

Learning task-agnostic dynamics models in high-dimensional observation s...
11/09/2019

Robo-PlaNet: Learning to Poke in a Day

Recently, the Deep Planning Network (PlaNet) approach was introduced as ...
01/25/2022

Tracking and Planning with Spatial World Models

We introduce a method for real-time navigation and tracking with differe...
10/23/2020

CLOUD: Contrastive Learning of Unsupervised Dynamics

Developing agents that can perform complex control tasks from high dimen...
09/29/2022

Learning Parsimonious Dynamics for Generalization in Reinforcement Learning

Humans are skillful navigators: We aptly maneuver through new places, re...
10/29/2019

Dynamics Learning with Cascaded Variational Inference for Multi-Step Manipulation

The fundamental challenge of planning for multi-step manipulation is to ...

Code Repositories

PlaNet

Deep Planning Network: Control from pixels by latent planning with learned dynamics


view repo

PlaNet_PyTorch

Unofficial re-implementation of "Learning Latent Dynamics for Planning from Pixels" (https://arxiv.org/abs/1811.04551 ) with PyTorch


view repo

cwvae

Clockwork Variational Autoencoder


view repo

cwvae-jax

Clockwork VAEs in JAX/Flax


view repo

softagent

Algorithms for deformable object manipulation benchmarked in SoftGym


view repo