Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation

10/03/2018
by   Dafni Antotsiou, et al.
6

Human hand actions are quite complex, especially when they involve object manipulation, mainly due to the high dimensionality of the hand and the vast action space that entails. Imitating those actions with dexterous hand models involves different important and challenging steps: acquiring human hand information, retargeting it to a hand model, and learning a policy from acquired data. In this work, we capture the hand information by using a state-of-the-art hand pose estimator. We tackle the retargeting problem from the hand pose to a 29 DoF hand model by combining inverse kinematics and PSO with a task objective optimisation. This objective encourages the virtual hand to accomplish the manipulation task, relieving the effect of the estimator's noise and the domain gap. Our approach leads to a better success rate in the grasping task compared to our inverse kinematics baseline, allowing us to record successful human demonstrations. Furthermore, we used these demonstrations to learn a policy network using generative adversarial imitation learning (GAIL) that is able to autonomously grasp an object in the virtual space.

READ FULL TEXT

page 7

page 11

research
03/24/2022

Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation

Optimizing behaviors for dexterous manipulation has been a longstanding ...
research
08/07/2020

Physics-Based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning

Dexterous manipulation of objects in virtual environments with our bare ...
research
04/23/2021

H2O: A Benchmark for Visual Human-human Object Handover Analysis

Object handover is a common human collaboration behavior that attracts a...
research
09/12/2022

HandMime: Sign Language Fingerspelling Acquisition via Imitation Learning

Learning fine-grained movements is among the most challenging topics in ...
research
12/26/2020

Stochastic Action Prediction for Imitation Learning

Imitation learning is a data-driven approach to acquiring skills that re...
research
12/09/2020

Toward an Affective Touch Robot: Subjective and Physiological Evaluation of Gentle Stroke Motion Using a Human-Imitation Hand

Affective touch offers positive psychological and physiological benefits...
research
07/26/2019

Learning to Solve a Rubik's Cube with a Dexterous Hand

We present a learning-based approach to solving a Rubik's cube with a mu...

Please sign up or login with your details

Forgot password? Click here to reset