From Motor Control to Team Play in Simulated Humanoid Football

05/25/2021
by   Siqi Liu, et al.
2

Intelligent behaviour in the physical world exhibits structure at multiple spatial and temporal scales. Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected to serve goals defined on much longer timescales, and in terms of relations that extend far beyond the body itself, ultimately involving coordination with other agents. Recent research in artificial intelligence has shown the promise of learning-based approaches to the respective problems of complex movement, longer-term planning and multi-agent coordination. However, there is limited research aimed at their integration. We study this problem by training teams of physically simulated humanoid avatars to play football in a realistic virtual environment. We develop a method that combines imitation learning, single- and multi-agent reinforcement learning and population-based training, and makes use of transferable representations of behaviour for decision making at different levels of abstraction. In a sequence of stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and play as a team, bridging the gap between low-level motor control at a timescale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds. We investigate the emergence of behaviours at different levels of abstraction, as well as the representations that underlie these behaviours using several analysis techniques, including statistics from real-world sports analytics. Our work constitutes a complete demonstration of integrated decision-making at multiple scales in a physically embodied multi-agent setting. See project video at https://youtu.be/KHMwq9pv7mg.

READ FULL TEXT

page 1

page 5

page 7

page 16

page 25

page 27

research
12/07/2019

Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery

Human players in professional team sports achieve high level coordinatio...
research
03/20/2018

Generative Multi-Agent Behavioral Cloning

We propose and study the problem of generative multi-agent behavioral cl...
research
10/04/2022

Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning

In cooperative multi-agent reinforcement learning, a team of agents work...
research
08/15/2023

Hierarchical generative modelling for autonomous robots

Humans can produce complex whole-body motions when interacting with thei...
research
06/22/2011

Towards Adjustable Autonomy for the Real World

Adjustable autonomy refers to entities dynamically varying their own aut...
research
09/10/2018

PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning

Multi-agent path finding (MAPF) is an essential component of many large-...
research
03/28/2023

FC Portugal 3D Simulation Team: Team Description Paper 2020

The FC Portugal 3D team is developed upon the structure of our previous ...

Please sign up or login with your details

Forgot password? Click here to reset