DeepAI
Log In Sign Up

Learning to Play Soccer From Scratch: Sample-Efficient Emergent Coordination through Curriculum-Learning and Competition

03/09/2021
by   Pavan Samtani, et al.
0

This work proposes a scheme that allows learning complex multi-agent behaviors in a sample efficient manner, applied to 2v2 soccer. The problem is formulated as a Markov game, and solved using deep reinforcement learning. We propose a basic multi-agent extension of TD3 for learning the policy of each player, in a decentralized manner. To ease learning, the task of 2v2 soccer is divided in three stages: 1v0, 1v1 and 2v2. The process of learning in multi-agent stages (1v1 and 2v2) uses agents trained on a previous stage as fixed opponents. In addition, we propose using experience sharing, a method that shares experience from a fixed opponent, trained in a previous stage, for training the agent currently learning, and a form of frame-skipping, to raise performance significantly. Our results show that high quality soccer play can be obtained with our approach in just under 40M interactions. A summarized video of the resulting game play can be found in https://youtu.be/f25l1j1U9RM.

READ FULL TEXT
08/18/2020

Learning Complex Multi-Agent Policies in Presence of an Adversary

In recent years, there has been some outstanding work on applying deep r...
09/13/2018

CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning

We propose CM3, a new deep reinforcement learning method for cooperative...
09/04/2019

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...
03/23/2020

Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning

In multi-agent games, the complexity of the environment can grow exponen...
02/19/2019

Emergent Coordination Through Competition

We study the emergence of cooperative behaviors in reinforcement learnin...
10/29/2020

Performance Indicators Contributing To Success At The Group And Play-Off Stages Of The 2019 Rugby World Cup

Performance indicators that contributed to success at the group stage an...
10/16/2020

Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

Pursuit-evasion is the problem of capturing mobile targets with one or m...