Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning

05/29/2023
by   Hiroshi Nakahara, et al.
0

Analysis of invasive sports such as soccer is challenging because the game situation changes continuously in time and space, and multiple agents individually recognize the game situation and make decisions. Previous studies using deep reinforcement learning have often considered teams as a single agent and valued the teams and players who hold the ball in each discrete event. Then it was challenging to value the actions of multiple players, including players far from the ball, in a spatiotemporally continuous state space. In this paper, we propose a method of valuing possible actions for on- and off-ball soccer players in a single holistic framework based on multi-agent deep reinforcement learning. We consider a discrete action space in a continuous state space that mimics that of Google research football and leverages supervised learning for actions in reinforcement learning. In the experiment, we analyzed the relationships with conventional indicators, season goals, and game ratings by experts, and showed the effectiveness of the proposed method. Our approach can assess how multiple players move continuously throughout the game, which is difficult to be discretized or labeled but vital for teamwork, scouting, and fan engagement.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/08/2017

Investigating Reinforcement Learning Agents for Continuous State Space Environments

Given an environment with continuous state spaces and discrete actions, ...
research
09/14/2021

Towards optimized actions in critical situations of soccer games with deep reinforcement learning

Soccer is a sparse rewarding game: any smart or careless action in criti...
research
08/16/2017

StarCraft II: A New Challenge for Reinforcement Learning

This paper introduces SC2LE (StarCraft II Learning Environment), a reinf...
research
11/13/2015

Deep Reinforcement Learning in Parameterized Action Space

Recent work has shown that deep neural networks are capable of approxima...
research
12/02/2018

Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strat...
research
02/23/2017

A Goal-Based Movement Model for Continuous Multi-Agent Tasks

Despite increasing attention paid to the need for fast, scalable methods...
research
09/15/2022

ProAPT: Projection of APT Threats with Deep Reinforcement Learning

The highest level in the Endsley situation awareness model is called pro...

Please sign up or login with your details

Forgot password? Click here to reset