The Right Variety: Improving Expressive Range Analysis with Metric Selection Methods

by   Oliver Withington, et al.

Expressive Range Analysis (ERA), an approach for visualising the output of Procedural Content Generation (PCG) systems, is widely used within PCG research to evaluate and compare generators, often to make comparative statements about their relative performance in terms of output diversity and search space exploration. Producing a standard ERA visualisation requires the selection of two metrics which can be calculated for all generated artefacts to be visualised. However, to our knowledge there are no methodologies or heuristics for justifying the selection of a specific metric pair over alternatives. Prior work has typically either made a selection based on established but unjustified norms, designer intuition, or has produced multiple visualisations across all possible pairs. This work aims to contribute to this area by identifying valuable characteristics of metric pairings, and by demonstrating that pairings that have these characteristics have an increased probability of producing an informative ERA projection of the underlying generator. We introduce and investigate three quantifiable selection criteria for assessing metric pairs, and demonstrate how these criteria can be operationalized to rank those available. Though this is an early exploration of the concept of quantifying the utility of ERA metric pairs, we argue that the approach explored in this paper can make ERA more useful and usable for both researchers and game designers.


page 8

page 9


An Axiomatic Analysis of Diversity Evaluation Metrics: Introducing the Rank-Biased Utility Metric

Many evaluation metrics have been defined to evaluate the effectiveness ...

Tabletop Roleplaying Games as Procedural Content Generators

Tabletop roleplaying games (TTRPGs) and procedural content generators ca...

Illuminating Diverse Neural Cellular Automata for Level Generation

We present a method of generating a collection of neural cellular automa...

Learning Controllable Content Generators

It has recently been shown that reinforcement learning can be used to tr...

Compressing and Comparing the Generative Spaces of Procedural Content Generators

The past decade has seen a rapid increase in the level of research inter...

Using Metrics Suites to Improve the Measurement of Privacy in Graphs

Social graphs are widely used in research (e.g., epidemiology) and busin...

Please sign up or login with your details

Forgot password? Click here to reset