A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation

10/11/2022
by   Lingfeng Tao, et al.
0

In-hand manipulation is challenging for a multi-finger robotic hand due to its high degrees of freedom and the complex interaction with the object. To enable in-hand manipulation, existing deep reinforcement learning based approaches mainly focus on training a single robot-structure-specific policy through the centralized learning mechanism, lacking adaptability to changes like robot malfunction. To solve this limitation, this work treats each finger as an individual agent and trains multiple agents to control their assigned fingers to complete the in-hand manipulation task cooperatively. We propose the Multi-Agent Global-Observation Critic and Local-Observation Actor (MAGCLA) method, where the critic can observe all agents' actions globally, and the actor only locally observes its neighbors' actions. Besides, conventional individual experience replay may cause unstable cooperation due to the asynchronous performance increment of each agent, which is critical for in-hand manipulation tasks. To solve this issue, we propose the Synchronized Hindsight Experience Replay (SHER) method to synchronize and efficiently reuse the replayed experience across all agents. The methods are evaluated in two in-hand manipulation tasks on the Shadow dexterous hand. The results show that SHER helps MAGCLA achieve comparable learning efficiency to a single policy, and the MAGCLA approach is more generalizable in different tasks. The trained policies have higher adaptability in the robot malfunction test compared to the baseline multi-agent and single-agent approaches.

READ FULL TEXT

page 1

page 4

page 6

research
07/05/2023

SACHA: Soft Actor-Critic with Heuristic-Based Attention for Partially Observable Multi-Agent Path Finding

Multi-Agent Path Finding (MAPF) is a crucial component for many large-sc...
research
09/13/2023

Stable In-hand Manipulation with Finger Specific Multi-agent Shadow Reward

Deep Reinforcement Learning has shown its capability to solve the high d...
research
11/28/2022

CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces

Multi-robot manipulation tasks involve various control entities that can...
research
09/13/2023

Curriculum-based Sensing Reduction in Simulation to Real-World Transfer for In-hand Manipulation

Simulation to Real-World Transfer allows affordable and fast training of...
research
01/03/2022

Asymptotic Convergence of Deep Multi-Agent Actor-Critic Algorithms

We present sufficient conditions that ensure convergence of the multi-ag...
research
06/28/2020

Inner Attention Modeling for Flexible Teaming of Heterogeneous Multi Robots Using Multi-Agent Reinforcement Learning

With the advantages of member diversity and team scale, heterogeneous mu...
research
02/12/2020

Inner Attention Supported Adaptive Cooperation for Heterogeneous Multi Robots Teaming based on Multi-agent Reinforcement Learning

Humans can selectively focus on different information based on different...

Please sign up or login with your details

Forgot password? Click here to reset