Experimental Studies in General Game Playing: An Experience Report

by   Jakub Kowalski, et al.

We describe nearly fifteen years of General Game Playing experimental research history in the context of reproducibility and fairness of comparisons between various GGP agents and systems designed to play games described by different formalisms. We think our survey may provide an interesting perspective of how chaotic methods were allowed when nothing better was possible. Finally, from our experience-based view, we would like to propose a few recommendations of how such specific heterogeneous branch of research should be handled appropriately in the future. The goal of this note is to point out common difficulties and problems in the experimental research in the area. We hope that our recommendations will help in avoiding them in future works and allow more fair and reproducible comparisons.



There are no comments yet.


page 1

page 2

page 3

page 4


General Video Game AI: a Multi-Track Framework for Evaluating Agents, Games and Content Generation Algorithms

General Video Game Playing (GVGP) aims at designing an agent that is cap...

Automated Game Design Learning

While general game playing is an active field of research, the learning ...

"Did You Hear That?" Learning to Play Video Games from Audio Cues

Game-playing AI research has focused for a long time on learning to play...

Leveling the Playing Field - Fairness in AI Versus Human Game Benchmarks

From the beginning if the history of AI, there has been interest in game...

Getting recommendation is not always better

We present an extended version of the Iterated Prisoner's Dilemma game i...

On tit for tat: Franceschini and Maisano versus ANVUR regarding the Italian research assessment exercise VQR 2011-2014

The response by Benedetto, Checchi, Graziosi & Malgarini (2017) (hereaft...

A note on the empirical comparison of RBG and Ludii

We present an experimental comparison of the efficiency of three General...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.