Co-design of Embodied Neural Intelligence via Constrained Evolution

by   Zhiquan Wang, et al.

We introduce a novel co-design method for autonomous moving agents' shape attributes and locomotion by combining deep reinforcement learning and evolution with user control. Our main inspiration comes from evolution, which has led to wide variability and adaptation in Nature and has the potential to significantly improve design and behavior simultaneously. Our method takes an input agent with optional simple constraints such as leg parts that should not evolve or allowed ranges of changes. It uses physics-based simulation to determine its locomotion and finds a behavior policy for the input design, later used as a baseline for comparison. The agent is then randomly modified within the allowed ranges creating a new generation of several hundred agents. The generation is trained by transferring the previous policy, which significantly speeds up the training. The best-performing agents are selected, and a new generation is formed using their crossover and mutations. The next generations are then trained until satisfactory results are reached. We show a wide variety of evolved agents, and our results show that even with only 10 changes, the overall performance of the evolved agents improves 50 significant changes to the initial design are allowed, our experiments' performance improves even more to 150 works on a single GPU and provides satisfactory results by training thousands of agents within one hour.


page 1

page 4

page 5

page 7

page 8


Learning through Probing: a decentralized reinforcement learning architecture for social dilemmas

Multi-agent reinforcement learning has received significant interest in ...

Kickstarting Deep Reinforcement Learning

We present a method for using previously-trained 'teacher' agents to kic...

Embodied Intelligence via Learning and Evolution

The intertwined processes of learning and evolution in complex environme...

Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments

Reproducibility in reinforcement learning is challenging: uncontrolled s...

Separation of Concerns in Reinforcement Learning

In this paper, we propose a framework for solving a single-agent task by...

Robust Dual View Depp Agent

Motivated by recent advance of machine learning using Deep Reinforcement...

How morphological development can guide evolution

Organisms result from multiple adaptive processes occurring and interact...

Please sign up or login with your details

Forgot password? Click here to reset