Few-Shot Goal Inference for Visuomotor Learning and Planning

by   Annie Xie, et al.

Reinforcement learning and planning methods require an objective or reward function that encodes the desired behavior. Yet, in practice, there is a wide range of scenarios where an objective is difficult to provide programmatically, such as tasks with visual observations involving unknown object positions or deformable objects. In these cases, prior methods use engineered problem-specific solutions, e.g., by instrumenting the environment with additional sensors to measure a proxy for the objective. Such solutions require a significant engineering effort on a per-task basis, and make it impractical for robots to continuously learn complex skills outside of laboratory settings. We aim to find a more general and scalable solution for specifying goals for robot learning in unconstrained environments. To that end, we formulate the few-shot objective learning problem, where the goal is to learn a task objective from only a few example images of successful end states for that task. We propose a simple solution to this problem: meta-learn a classifier that can recognize new goals from a few examples. We show how this approach can be used with both model-free reinforcement learning and visual model-based planning and show results in three domains: rope manipulation from images in simulation, visual navigation in a simulated 3D environment, and object arrangement into user-specified configurations on a real robot.


page 6

page 7

page 8

page 13


End-to-End Robotic Reinforcement Learning without Reward Engineering

The combination of deep neural network models and reinforcement learning...

Deep Reinforcement Learning with Successor Features for Navigation across Similar Environments

In this paper we consider the problem of robot navigation in simple maze...

Model-Based Visual Planning with Self-Supervised Functional Distances

A generalist robot must be able to complete a variety of tasks in its en...

Unsupervised Visuomotor Control through Distributional Planning Networks

While reinforcement learning (RL) has the potential to enable robots to ...

Skew-Fit: State-Covering Self-Supervised Reinforcement Learning

In standard reinforcement learning, each new skill requires a manually-d...

Reinforcement Learning without Ground-Truth State

To perform robot manipulation tasks, a low dimension state of the enviro...

SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning

Model-based reinforcement learning algorithms are typically more sample ...

Please sign up or login with your details

Forgot password? Click here to reset