Universal Planning Networks

04/02/2018
by   Aravind Srinivas, et al.
0

A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization. To this end, we introduce universal planning networks (UPN). UPNs embed differentiable planning within a goal-directed policy. This planning computation unrolls a forward model in a latent space and infers an optimal action plan through gradient descent trajectory optimization. The plan-by-gradient-descent process and its underlying representations are learned end-to-end to directly optimize a supervised imitation learning objective. We find that the representations learned are not only effective for goal-directed visual imitation via gradient-based trajectory optimization, but can also provide a metric for specifying goals using images. The learned representations can be leveraged to specify distance-based rewards to reach new target states for model-free reinforcement learning, resulting in substantially more effective learning when solving new tasks described via image-based goals. We were able to achieve successful transfer of visuomotor planning strategies across robots with significantly different morphologies and actuation capabilities.

READ FULL TEXT

page 6

page 9

page 10

research
03/08/2019

Dyna-AIL : Adversarial Imitation Learning by Planning

Adversarial methods for imitation learning have been shown to perform we...
research
06/29/2017

Path Integral Networks: End-to-End Differentiable Optimal Control

In this paper, we introduce Path Integral Networks (PI-Net), a recurrent...
research
10/05/2021

Procedure Planning in Instructional Videosvia Contextual Modeling and Model-based Policy Learning

Learning new skills by observing humans' behaviors is an essential capab...
research
05/19/2017

Model-Based Planning in Discrete Action Spaces

Planning actions using learned and differentiable forward models of the ...
research
12/28/2016

The Predictron: End-To-End Learning and Planning

One of the key challenges of artificial intelligence is to learn models ...
research
04/14/2017

Optimizing Differentiable Relaxations of Coreference Evaluation Metrics

Coreference evaluation metrics are hard to optimize directly as they are...
research
10/07/2021

Goal-Directed Design Agents: Integrating Visual Imitation with One-Step Lookahead Optimization for Generative Design

Engineering design problems often involve large state and action spaces ...

Please sign up or login with your details

Forgot password? Click here to reset