DeepAI AI Chat
Log In Sign Up

Learning Cooperative Dynamic Manipulation Skills from Human Demonstration Videos

by   Francesco Iodice, et al.
Xi'an Jiaotong University
Istituto Italiano di Tecnologia

This article proposes a method for learning and robotic replication of dynamic collaborative tasks from offline videos. The objective is to extend the concept of learning from demonstration (LfD) to dynamic scenarios, benefiting from widely available or easily producible offline videos. To achieve this goal, we decode important dynamic information, such as the Configuration Dependent Stiffness (CDS), which reveals the contribution of arm pose to the arm endpoint stiffness, from a three-dimensional human skeleton model. Next, through encoding of the CDS via Gaussian Mixture Model (GMM) and decoding via Gaussian Mixture Regression (GMR), the robot's Cartesian impedance profile is estimated and replicated. We demonstrate the proposed method in a collaborative sawing task with leader-follower structure, considering environmental constraints and dynamic uncertainties. The experimental setup includes two Panda robots, which replicate the leader-follower roles and the impedance profiles extracted from a two-persons sawing video.


page 2

page 3

page 6

page 10

page 11


Active Improvement of Control Policies with Bayesian Gaussian Mixture Model

Learning from demonstration (LfD) is an intuitive framework allowing non...

Task-Adaptive Robot Learning from Demonstration under Replication with Gaussian Process Models

Learning from Demonstration (LfD) is a paradigm that allows robots to le...

Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models

A core challenge for an autonomous agent acting in the real world is to ...

Augmentation for Learning From Demonstration with Environmental Constraints

We introduce a Learning from Demonstration (LfD) approach for contact-ri...

BiRP: Learning Robot Generalized Bimanual Coordination using Relative Parameterization Method on Human Demonstration

Human bimanual manipulation can perform more complex tasks than a simple...

A Hybrid Learning and Optimization Framework to Achieve Physically Interactive Tasks with Mobile Manipulators

This paper proposes a hybrid learning and optimization framework for mob...