Deep Generative Multi-Agent Imitation Model as a Computational Benchmark for Evaluating Human Performance in Complex Interactive Tasks: A Case Study in Football

by   Chaoyi Gu, et al.

Evaluating the performance of human is a common need across many applications, such as in engineering and sports. When evaluating human performance in completing complex and interactive tasks, the most common way is to use a metric having been proved efficient for that context, or to use subjective measurement techniques. However, this can be an error prone and unreliable process since static metrics cannot capture all the complex contexts associated with such tasks and biases exist in subjective measurement. The objective of our research is to create data-driven AI agents as computational benchmarks to evaluate human performance in solving difficult tasks involving multiple humans and contextual factors. We demonstrate this within the context of football performance analysis. We train a generative model based on Conditional Variational Recurrent Neural Network (VRNN) Model on a large player and ball tracking dataset. The trained model is used to imitate the interactions between two teams and predict the performance from each team. Then the trained Conditional VRNN Model is used as a benchmark to evaluate team performance. The experimental results on Premier League football dataset demonstrates the usefulness of our method to existing state-of-the-art static metric used in football analytics.


page 1

page 6

page 7


Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi

Deep reinforcement learning has generated superhuman AI in competitive g...

Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems

Explainable artificially intelligent (XAI) systems form part of sociotec...

Warmth and competence in human-agent cooperation

Interaction and cooperation with humans are overarching aspirations of a...

Benchmarking LLM powered Chatbots: Methods and Metrics

Autonomous conversational agents, i.e. chatbots, are becoming an increas...

Embedding Contextual Information through Reward Shaping in Multi-Agent Learning: A Case Study from Google Football

Artificial Intelligence has been used to help human complete difficult t...

Building Better Human-Agent Teams: Tradeoffs in Helpfulness and Humanness in Voice

We manipulate the helpfulness and voice type of a voice-only agent teamm...

Qualitative Prediction of Multi-Agent Spatial Interactions

Deploying service robots in our daily life, whether in restaurants, ware...

Please sign up or login with your details

Forgot password? Click here to reset