DeepAI AI Chat
Log In Sign Up

Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning

by   Satoshi Kataoka, et al.

Most successes in robotic manipulation have been restricted to single-arm gripper robots, whose low dexterity limits the range of solvable tasks to pick-and-place, inser-tion, and object rearrangement. More complex tasks such as assembly require dual and multi-arm platforms, but entail a suite of unique challenges such as bi-arm coordination and collision avoidance, robust grasping, and long-horizon planning. In this work we investigate the feasibility of training deep reinforcement learning (RL) policies in simulation and transferring them to the real world (Sim2Real) as a generic methodology for obtaining performant controllers for real-world bi-manual robotic manipulation tasks. As a testbed for bi-manual manipulation, we develop the U-Shape Magnetic BlockAssembly Task, wherein two robots with parallel grippers must connect 3 magnetic blocks to form a U-shape. Without manually-designed controller nor human demonstrations, we demonstrate that with careful Sim2Real considerations, our policies trained with RL in simulation enable two xArm6 robots to solve the U-shape assembly task with a success rate of above90 real hardware without any additional real-world fine-tuning. Through careful ablations,we highlight how each component of the system is critical for such simple and successful policy learning and transfer,including task specification, learning algorithm, direct joint-space control, behavior constraints, perception and actuation noises, action delays and action interpolation. Our results present a significant step forward for bi-arm capability on real hardware, and we hope our system can inspire future research on deep RL and Sim2Real transfer of bi-manualpolicies, drastically scaling up the capability of real-world robot manipulators.


page 1

page 3


Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning

Most successes in robotic manipulation have been restricted to single-ar...

FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulation

Reinforcement learning (RL), imitation learning (IL), and task and motio...

Bi-Touch: Bimanual Tactile Manipulation with Sim-to-Real Deep Reinforcement Learning

Bimanual manipulation with tactile feedback will be key to human-level r...

IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality

Robotic assembly is a longstanding challenge, requiring contact-rich int...

Flexible Robotic Grasping with Sim-to-Real Transfer based Reinforcement Learning

Robotic manipulation requires a highly flexible and compliant system. Ta...

Learning to Centralize Dual-Arm Assembly

Even though industrial manipulators are widely used in modern manufactur...

Robust Relative Hand Placement For Bi-Manual Tasks

In many bi-manual robotic tasks, like peg-in-a-hole assembly, the succes...