Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning

11/04/2021
by   Wenlong Huang, et al.
0

Dexterous manipulation of arbitrary objects, a fundamental daily task for humans, has been a grand challenge for autonomous robotic systems. Although data-driven approaches using reinforcement learning can develop specialist policies that discover behaviors to control a single object, they often exhibit poor generalization to unseen ones. In this work, we show that policies learned by existing reinforcement learning algorithms can in fact be generalist when combined with multi-task learning and a well-chosen object representation. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms the single-object specialist policies on both training as well as held-out test objects. Video results at https://huangwl18.github.io/geometry-dex

READ FULL TEXT

page 1

page 2

page 5

research
02/03/2018

Multi-task Learning for Continuous Control

Reliable and effective multi-task learning is a prerequisite for the dev...
research
10/24/2019

Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

Meta-reinforcement learning algorithms can enable robots to acquire new ...
research
03/18/2021

Generalizing Object-Centric Task-Axes Controllers using Keypoints

To perform manipulation tasks in the real world, robots need to operate ...
research
12/01/2021

Tool as Embodiment for Recursive Manipulation

Humans and many animals exhibit a robust capability to manipulate divers...
research
08/22/2022

Detect Hate Speech in Unseen Domains using Multi-Task Learning: A Case Study of Political Public Figures

Automatic identification of hateful and abusive content is vital in comb...
research
06/07/2020

Multi-Task Reinforcement Learning based Mobile Manipulation Control for Dynamic Object Tracking and Grasping

Agile control of mobile manipulator is challenging because of the high c...
research
09/24/2019

Accept Synthetic Objects as Real: End-to-End Training of Attentive Deep Visuomotor Policies for Manipulation in Clutter

Recent research demonstrated that it is feasible to end-to-end train mul...

Please sign up or login with your details

Forgot password? Click here to reset