Goal-Aware Prediction: Learning to Model What Matters

07/14/2020
by   Suraj Nair, et al.
13

Learned dynamics models combined with both planning and policy learning algorithms have shown promise in enabling artificial agents to learn to perform many diverse tasks with limited supervision. However, one of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model (future state reconstruction), and that of the downstream planner or policy (completing a specified task). This issue is exacerbated by vision-based control tasks in diverse real-world environments, where the complexity of the real world dwarfs model capacity. In this paper, we propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space, resulting in a learning objective that more closely matches the downstream task. Further, we do so in an entirely self-supervised manner, without the need for a reward function or image labels. We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.

READ FULL TEXT

page 5

page 7

page 8

page 9

page 14

research
12/30/2020

Model-Based Visual Planning with Self-Supervised Functional Distances

A generalist robot must be able to complete a variety of tasks in its en...
research
02/11/2020

Objective Mismatch in Model-based Reinforcement Learning

Model-based reinforcement learning (MBRL) has been shown to be a powerfu...
research
10/22/2020

Batch Exploration with Examples for Scalable Robotic Reinforcement Learning

Learning from diverse offline datasets is a promising path towards learn...
research
12/04/2020

Planning from Pixels using Inverse Dynamics Models

Learning task-agnostic dynamics models in high-dimensional observation s...
research
08/25/2019

Dynamics-aware Embeddings

In this paper we consider self-supervised representation learning to imp...
research
09/12/2019

Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation

Video prediction models combined with planning algorithms have shown pro...
research
06/02/2020

NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spaces

Learning low-dimensional latent state space dynamics models has been a p...

Please sign up or login with your details

Forgot password? Click here to reset