Hybrid Learning- and Model-Based Planning and Control of In-Hand Manipulation

by   Rana Soltani-Zarrin, et al.

This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifingered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the grasp sequence is planned online by a learning-based policy to adapt to variations, the trajectory planner and the low-level controller for object tracking and contact force control are exclusively model-based to robustly realize the plan. By infusing the knowledge about the physics of the problem and the low-level controller into the grasp planner, it learns to successfully generate grasps similar to those generated by model-based optimization approaches, obviating the high computation cost of online running of such methods to account for variations. By performing experiments in physics simulation for realistic tool use scenarios, we show the success of our method on different tool-use tasks and dexterous hand models. Additionally, we show that this hybrid method offers more robustness to trajectory and task variations compared to a model-based method.


page 1

page 5


Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation

This paper explores the problem of autonomous, in-hand regrasping--the p...

Learning Hierarchical Control for Robust In-Hand Manipulation

Robotic in-hand manipulation has been a long-standing challenge due to t...

Towards Robust Autonomous Grasping with Reflexes Using High-Bandwidth Sensing and Actuation

Modern robotic manipulation systems fall short of human manipulation ski...

Online augmentation of learned grasp sequence policies for more adaptable and data-efficient in-hand manipulation

When using a tool, the grasps used for picking it up, reposing, and hold...

Complex In-Hand Manipulation via Compliance-Enabled Finger Gaiting and Multi-Modal Planning

Constraining contacts to remain fixed on an object during manipulation l...

Combining Coarse and Fine Physics for Manipulation using Parallel-in-Time Integration

We present a method for fast and accurate physics-based predictions duri...

Force Feedback Control For Dexterous Robotic Hands Using Conditional Postural Synergies

We present a force feedback controller for a dexterous robotic hand equi...

Please sign up or login with your details

Forgot password? Click here to reset