Emergent Complexity via Multi-Agent Competition

by   Trapit Bansal, et al.
University of Massachusetts Amherst

Reinforcement learning algorithms can train agents that solve problems in complex, interesting environments. Normally, the complexity of the trained agent is closely related to the complexity of the environment. This suggests that a highly capable agent requires a complex environment for training. In this paper, we point out that a competitive multi-agent environment trained with self-play can produce behaviors that are far more complex than the environment itself. We also point out that such environments come with a natural curriculum, because for any skill level, an environment full of agents of this level will have the right level of difficulty. This work introduces several competitive multi-agent environments where agents compete in a 3D world with simulated physics. The trained agents learn a wide variety of complex and interesting skills, even though the environment themselves are relatively simple. The skills include behaviors such as running, blocking, ducking, tackling, fooling opponents, kicking, and defending using both arms and legs. A highlight of the learned behaviors can be found here: https://goo.gl/eR7fbX


Learning Complex Multi-Agent Policies in Presence of an Adversary

In recent years, there has been some outstanding work on applying deep r...

Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real

Manipulation and locomotion are closely related problems that are often ...

Emergent Tool Use From Multi-Agent Autocurricula

Through multi-agent competition, the simple objective of hide-and-seek, ...

Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space

Learning competitive behaviors in multi-agent settings such as racing re...

Eco-evolutionary Dynamics of Non-episodic Neuroevolution in Large Multi-agent Environments

Neuroevolution (NE) has recently proven a competitive alternative to lea...

Flatland: a Lightweight First-Person 2-D Environment for Reinforcement Learning

We propose Flatland, a simple, lightweight environment for fast prototyp...

Learning to Play with Intrinsically-Motivated Self-Aware Agents

Infants are experts at playing, with an amazing ability to generate nove...

Please sign up or login with your details

Forgot password? Click here to reset