DeepAI AI Chat
Log In Sign Up

ManiSkill: Learning-from-Demonstrations Benchmark for Generalizable Manipulation Skills

by   Tongzhou Mu, et al.
University of California, San Diego

Learning generalizable manipulation skills is central for robots to achieve task automation in environments with endless scene and object variations. However, existing robot learning environments are limited in both scale and diversity of 3D assets (especially of articulated objects), making it difficult to train and evaluate the generalization ability of agents over novel objects. In this work, we focus on object-level generalization and propose SAPIEN Manipulation Skill Benchmark (abbreviated as ManiSkill), a large-scale learning-from-demonstrations benchmark for articulated object manipulation with 3D visual input (point cloud and RGB-D image). ManiSkill supports object-level variations by utilizing a rich and diverse set of articulated objects, and each task is carefully designed for learning manipulations on a single category of objects. We equip ManiSkill with a large number of high-quality demonstrations to facilitate learning-from-demonstrations approaches and perform evaluations on baseline algorithms. We believe that ManiSkill can encourage the robot learning community to explore more on learning generalizable object manipulation skills.


page 1

page 5


Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs

This paper presents a new technique for learning category-level manipula...

Deep Object-Centric Representations for Generalizable Robot Learning

Robotic manipulation in complex open-world scenarios requires both relia...

Vision-based Robot Manipulation Learning via Human Demonstrations

Vision-based learning methods provide promise for robots to learn comple...

Reusable neural skill embeddings for vision-guided whole body movement and object manipulation

Both in simulation settings and robotics, there is an ambition to produc...

Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

A robot operating in a household environment will see a wide range of un...

Robobarista: Object Part based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds

There is a large variety of objects and appliances in human environments...

Holo-Dex: Teaching Dexterity with Immersive Mixed Reality

A fundamental challenge in teaching robots is to provide an effective in...