Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning

Assembly of multi-part physical structures is both a valuable end product for autonomous robotics, as well as a valuable diagnostic task for open-ended training of embodied intelligent agents. We introduce a naturalistic physics-based environment with a set of connectable magnet blocks inspired by children's toy kits. The objective is to assemble blocks into a succession of target blueprints. Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter. Furthermore, assembly stresses agents' multi-step planning, physical reasoning, and bimanual coordination. We find that the combination of large-scale reinforcement learning and graph-based policies – surprisingly without any additional complexity – is an effective recipe for training agents that not only generalize to complex unseen blueprints in a zero-shot manner, but even operate in a reset-free setting without being trained to do so. Through extensive experiments, we highlight the importance of large-scale training, structured representations, contributions of multi-task vs. single-task learning, as well as the effects of curriculums, and discuss qualitative behaviors of trained agents.

READ FULL TEXT
research
04/05/2019

Structured agents for physical construction

Physical construction -- the ability to compose objects, subject to phys...
research
03/10/2022

Learn2Assemble with Structured Representations and Search for Robotic Architectural Construction

Autonomous robotic assembly requires a well-orchestrated sequence of hig...
research
06/01/2020

Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas

We demonstrate a reinforcement learning agent which uses a compositional...
research
10/15/2022

PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale

The predictive information, the mutual information between the past and ...
research
10/19/2019

A Structured Prediction Approach for Generalization in Cooperative Multi-Agent Reinforcement Learning

Effective coordination is crucial to solve multi-agent collaborative (MA...
research
10/27/2020

One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL

While reinforcement learning algorithms can learn effective policies for...
research
10/03/2020

Actionet: An Interactive End-To-End Platform For Task-Based Data Collection And Augmentation In 3D Environment

The problem of task planning for artificial agents remains largely unsol...

Please sign up or login with your details

Forgot password? Click here to reset