RoboNet: Large-Scale Multi-Robot Learning

by   Sudeep Dasari, et al.

Robot learning has emerged as a promising tool for taming the complexity and diversity of the real world. Methods based on high-capacity models, such as deep networks, hold the promise of providing effective generalization to a wide range of open-world environments. However, these same methods typically require large amounts of diverse training data to generalize effectively. In contrast, most robotic learning experiments are small-scale, single-domain, and single-robot. This leads to a frequent tension in robotic learning: how can we learn generalizable robotic controllers without having to collect impractically large amounts of data for each separate experiment? In this paper, we propose RoboNet, an open database for sharing robotic experience, which provides an initial pool of 15 million video frames, from 7 different robot platforms, and study how it can be used to learn generalizable models for vision-based robotic manipulation. We combine the dataset with two different learning algorithms: visual foresight, which uses forward video prediction models, and supervised inverse models. Our experiments test the learned algorithms' ability to work across new objects, new tasks, new scenes, new camera viewpoints, new grippers, or even entirely new robots. In our final experiment, we find that by pre-training on RoboNet and fine-tuning on data from a held-out Franka or Kuka robot, we can exceed the performance of a robot-specific training approach that uses 4x-20x more data. For videos and data, see the project webpage:


page 2

page 5

page 7

page 13


Efficient Adaptation for End-to-End Vision-Based Robotic Manipulation

One of the great promises of robot learning systems is that they will be...

Know Thyself: Transferable Visuomotor Control Through Robot-Awareness

Training visuomotor robot controllers from scratch on a new robot typica...

Deep Visual Foresight for Planning Robot Motion

A key challenge in scaling up robot learning to many skills and environm...

Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trials

Recent progress in deep learning highlights the tremendous potential of ...

Where is my hand? Deep hand segmentation for visual self-recognition in humanoid robots

The ability to distinguish between the self and the background is of par...

Real-world Transfer of Evolved Artificial Immune System Behaviours between Small and Large Scale Robotic Platforms

In mobile robotics, a solid test for adaptation is the ability of a cont...

Pave the Way to Grasp Anything: Transferring Foundation Models for Universal Pick-Place Robots

Improving the generalization capabilities of general-purpose robotic age...

Please sign up or login with your details

Forgot password? Click here to reset