Log In Sign Up

Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition

by   Pavan Samtani, et al.

This work proposes a scheme that allows learning complex multi-agent behaviors in a sample efficient manner, applied to 2v2 soccer. The problem is formulated as a Markov game, and solved using deep reinforcement learning. We propose a basic multi-agent extension of TD3 for learning the policy of each player, in a decentralized manner. To ease learning, the task of 2v2 soccer is divided in three stages: 1v0, 1v1 and 2v2. The process of learning in multi-agent stages (1v1 and 2v2) uses agents trained on a previous stage as fixed opponents. In addition, we propose using experience sharing, a method that shares experience from a fixed opponent, trained in a previous stage, for training the agent currently learning, and a form of frame-skipping, to raise performance significantly. Our results show that high quality soccer play can be obtained with our approach in just under 40M interactions. A summarized video of the resulting game play can be found in


Learning Complex Multi-Agent Policies in Presence of an Adversary

In recent years, there has been some outstanding work on applying deep r...

CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning

We propose CM3, a new deep reinforcement learning method for cooperative...

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...

Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning

In multi-agent games, the complexity of the environment can grow exponen...

Emergent Coordination Through Competition

We study the emergence of cooperative behaviors in reinforcement learnin...

Performance Indicators Contributing To Success At The Group And Play-Off Stages Of The 2019 Rugby World Cup

Performance indicators that contributed to success at the group stage an...

Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

Pursuit-evasion is the problem of capturing mobile targets with one or m...