The Right Variety: Improving Expressive Range Analysis with Metric Selection Methods
share
Expressive Range Analysis (ERA), an approach for visualising the output of Procedural Content Generation (PCG) systems, is widely used within PCG research to evaluate and compare generators, often to make comparative statements about their relative performance in terms of output diversity and search space exploration. Producing a standard ERA visualisation requires the selection of two metrics which can be calculated for all generated artefacts to be visualised. However, to our knowledge there are no methodologies or heuristics for justifying the selection of a specific metric pair over alternatives. Prior work has typically either made a selection based on established but unjustified norms, designer intuition, or has produced multiple visualisations across all possible pairs. This work aims to contribute to this area by identifying valuable characteristics of metric pairings, and by demonstrating that pairings that have these characteristics have an increased probability of producing an informative ERA projection of the underlying generator. We introduce and investigate three quantifiable selection criteria for assessing metric pairs, and demonstrate how these criteria can be operationalized to rank those available. Though this is an early exploration of the concept of quantifying the utility of ERA metric pairs, we argue that the approach explored in this paper can make ERA more useful and usable for both researchers and game designers.
READ FULL TEXT