Games are generally structured towards being reactive to the player to provide an immersive experience. For example, difficulty may scale as the player’s character (PC) grows in strength or the environment and/or story may change as the PC makes in-game decisions. Such updates may be considered to be adaptations to enrich gameplay. In the case of an arcade-style game (i.e., where high score is the goal), such changes include reactions to the player’s skill level, managing performance concerns, and optimizing overall game flow. This paper introduces a technique for incorporating software artifacts (e.g., requirements, goals, etc.) as first-class citizens in an adaptive gameplay loop. We apply this technique to our existing open-source game project to demonstrate its effectiveness.
Self-adaptive systems (SAS) are an approach for enabling reconfiguration at run time in systems facing uncertainty (kephart.2003; mckinley.2004), where the MAPE-K feedback loop (kephart.2003) is one common technique for enabling adaptation. For instance, a system may change its configuration, algorithm, or decision-making engine as adversity manifests from combinations of system or environmental uncertainty (e.g., human interaction, unexpected external conditions, etc.). In terms of games, such configurations may include the number and intelligence of enemy characters, distribution of item pickups in relation to player level, and maintaining an appropriate frame rate as the number of instantiated objects increase. Moreover, software engineering techniques such as goal modeling (lamsweerde.1998) and run-time requirements monitoring (degrandis.2009) can enable introspection that supports self-reconfiguration strategies (bowers.2020), however these self-reconfiguration techniques have not been intrinsically applied to a game loop.
This paper describes our application of a MAPE-K feedback loop to a game designed as a creative coding learning experience for students. We generated the necessary software artifacts (i.e., a goal model (lamsweerde.2009) with utility functions (degrandis.2009)), updated existing source code with a self-adaptive overlay, and added instrumentation for monitoring the performance of the overall system with respect to our defined metrics.
Our initial results demonstrate the feasibility of applying self-adaptation to a gameplay loop. The remainder of this paper is structured as follows. Section 2 discusses relevant background information on creative coding (focusing on our motivating example), SASs, and goal modeling. Section 3 then discusses our approach for modeling a game and its software artifacts as a self-adaptive feedback loop and presents our initial results. Lastly, Section 4 summarizes our results and presents future directions.
2. Background and Related Work
This section describes our motivating example, SASs, and goal modeling. For each sub-section we also highlight related work.
2.1. Creative Coding (Motivating Example)
Motivating example - feesh: feesh is a relatively simple web-based game where the intention is to grow by eating smaller objects. This project was written in p5.js and can run in any browser without the need for a server. Additionally, the source code is available on GitHub.222https://github.com/efredericks/feesh.
The gameplay loop for feesh is based on Fishy, a classic Flash game where the intent is to avoid large fish and eat smaller fish.333See https://www.silvergames.com/en/fishy. The intent of feesh was to replicate the core mechanics of Fishy while introducing self-adaptation into the core gameplay loop. For additional processing complexity, all objects are formed via noise loops to give a “wobbly” impression, based upon the TheCodingTrain’s Blobby! video (shiffman.2016). Collision detection is simplified to be circle-circle collision between all entities.
Creative coding is typically structured around introducing students to computer science concepts via generative art or simplified game design as a way for students to graphically visualize difficult problems (peppler.2005; bergstrom.2015), with popular textbooks including The Nature of Code (shiffman.2012) and Generative Art: A Practical Guide using Processing (pearson.2011). Recently, creative coding has been applied to Internet of Things applications to further express art in real-world settings (vestergaard.2017). However, many papers centered around creative coding focus on teaching coding concepts, whereas we posit learning can be extended to additionally comprise complex software engineering concepts (where this argument will be explored in future works). Similarly, Bucchiarone et al. discuss gamification as a means for enhancing user engagement (bucchiarone.2019; bucchiarone.2021), however for the purposes of this paper we aim to enhance gameplay via self-adaptation in contrast to adding gameplay elements an existing application.
2.2. Self-Adaptive Systems
SASs provide an approach for self-reconfiguring (e.g., configuration, algorithm, etc.) at run time (mckinley.2004; oreizy.1999) in response to changing environmental and system conditions to continuously satisfy key objectives (cheng.2009; whittle.2009). The requirements for the system itself may also change over time, potentially necessitating updates in terms of patches, bug fixes, or configuration change. For the purposes of this paper we consider an SAS to comprise a set of configurable states connected via adaptive logic (zhang.2006). While there exist multiple approaches for enabling self-adaptation, we will follow the MAPE-K (Monitor-Analyze-Plan-Execute-Knowledge) feedback architecture (kephart.2003). We next describe our motivating example in the context of MAPE-K to highlight each aspect of its architecture.
Monitor: An SAS must monitor itself and its environment for decision-making purposes. We specified key metrics as relevant to the adaptive properties of our game engine, with internal software monitors being responsible for watching the state of each metric. Specifically, we manually identified which aspects of our game engine were amenable to adaptation (e.g., player size, enemy count) and which metrics might impact those aspects (e.g., FPS, execution time) A subset of these metrics, along with thresholds and reconfiguration strategies, are listed as follows in Table 1. Note, all values found in Table 1 were empirically derived.
|Monitor||Threshold||Reconfiguration Strategy||Affected Goal(s)|
|Frame rate (FPS)||¿=30||[Reduce enemy count, Disable enemy-enemy collision]||(B), (D), (E)|
|Playability||Maximize||[Reduce player size, Reduce enemy count]||(E), (F)|
|Number of enemies on screen||Maximize||N/A||(B), (E)|
|Enemy-enemy collision enabled?||T—F||N/A||(B), (D)|
|Execution time||Maximize||[Reduce player size, Reduce enemy count]||(A), (B), (C), (D), (E), (F)|
|Random event||chance||[Increase enemy count]||(A), (C), (E)|
Note that, for the purposes of this paper we are mainly focusing on features that directly impact the player during gameplay, however for future work we could examine past-game history (e.g., player behaviors, enemy encounters, etc.) to determine if adaptations are warranted.
Analyze: Each monitored attribute is then analyzed to determine if a violation has occurred or a threshold has been exceeded, thus necessitating a reconfiguration.444In other systems, continuous learning may be implemented to make predictions, however self-reconfiguration in feesh is reactive. For this project, we defined thresholds for each monitored attribute to determine if a reconfiguration is warranted. For example, if the frame rate (i.e., frames per second, or FPS) falls below then an adaptation is necessary to ensure the player has a smooth experience.
Plan: Based on identified adaptation requirements the Plan phase will select a reconfiguration strategy. Depending on the complexity of the system and feedback loop, this phase may involve the generation of many possible adaptations. For example, when the FPS drops below the system will have the choice to remove the enemy-enemy collision (i.e., calculating collisions between enemies to induce “bouncing”) or to remove a random number of enemies to reduce overall processing load.
Execute: The system will then execute the adaptation strategy based on monitored data and the decisions made in the Analysis and Plan phases. In the case of feesh, self-adaptation can comprise changes in complexity (e.g., increase the number of enemies given acceptable monitored performance), reduce the size of entities (e.g., scale down player size to improve playability), or change collision detection between enemies (e.g., removing enemy-enemy collision to improve performance). The Execute phase then directly applies the reconfiguration as needed.
Knowledge: For simplicity, knowledge of all aspects of the game and its self-adaptation mechanics are globally-accessible to all components of the MAPE-K loop.
Uncertainty: For the purposes of this paper, we will consider uncertainty to reflect human interaction (i.e., the player), the capabilities of the player’s computer (e.g., browser choice, hardware specifications, frame rate, etc.), and random behaviors implemented within the feesh game engine. The combination of these types of uncertainty can result in different gameplay experiences, and as such, feesh has been modeled as self-adaptive.
Previously, the MAPE-K loop has been applied to a gaming application by Yamagata et al. (yamagata.2019), however their application focused mainly on the latency issue in multiplayer games, whereas our application focuses on including adaptation as part of the intrinsic game mechanics. Cámara et al. investigated how an SAS can optimize network latency via gamification and by modeling it as a stochastic multiplayer game (camara.2016). Other methods of adaptation, among many, include the use of dynamic software product lines to update behaviors at run time (ayala.2021), Bayesian optimization with fuzzy systems (pirovano.2012)
, and artificial intelligence(swiechowski.2013).
2.3. Goal Modeling
Goal-oriented requirements engineering (GORE) uses goals (i.e., high-level objectives and/or requirements) to model system behaviors and objectives (lamsweerde.2009). GORE models a system-to-be via an acyclic graph comprising goals, refinements, requirements/expectations (i.e., leaf-level goals), and agents. Knowledge Acquisition in Automated Specification (KAOS) extends GORE via additional AND- and OR-refinements (lamsweerde.2009; dardenne.1993) (e.g., Goals (A) and (B) in Figure 2, respectively).. AND-refined goals are satisfied only when all subgoals are satisfied and OR-refined goals are satisfied when at minimum one of its subgoals are satisfied. Figure 2 presents a KAOS (lamsweerde.2009) goal model for feesh, where KAOS is one approach to goal modeling.555A similar approach for goal modeling is iStar (yu.1997).
The main goal of feesh is to extend time played (similar to an arcade game) (Goal (A)). To satisfy this goal, the game must be at an acceptable frame rate (Goal (B)) while ensuring it remains playable (Goal (C)). Goals (D) and (E) attempt to balance FPS by managing collision detection and the number of enemies on screen, respectively. Goals (F), (G), and (H) support Goal (C) by keeping the player at a manageable size while ensuring the score continually increases (to maximize player satisfaction). Moreover, Goals (I) and (J) additionally support (C) by enabling player and enemy control, respectively.
Goal (A) represents an AND-satisfied goal (i.e., Goals (B) and (C) must be satisfied for (A) to be satisfied) and Goal (B) represents an OR-satisfied goal (i.e., either Goals (D) or (E) must be satisfied for (B) to be satisfied). Moreover, goals designated as Maintain are considered to be invariant and those designated as Achieve are considered as non-invariant, where invariant goals cannot tolerate a violation and non-invariant goals can temporarily accept a transient violation.
Note that maximizing the number of enemies on screen while setting enemy-enemy collision to be enabled are competing concerns, as the collision detection algorithm implemented may significantly reduce frame rate. We would like to note that more “intelligent” collision detection algorithms can be implemented to minimize this issue (e.g., executed in parallel on a GPU, using local search/partitioning, etc.), however for the purposes of this paper a simpler implementation was preferred to demonstrate the issue.
We next describe utility functions, the approach we use for determining goal satisfaction at run time.
Utility Functions: Utility functions are mathematical formulae that have been used to quantify software requirement satisfaction (degrandis.2009; ramirez.2011; walsh.2004). In the context of an SAS, a utility function can serve as a metric for deciding if a reconfiguration is necessary (e.g., in the event that a goal is violated or under-performing). For example, a utility function can be derived for Goal (D) in Figure 2 as follows in Equation 1:
, where a value of indicates a violation, indicates satisfaction, and any value in between denotes the degree of satisficement. For this project, each utility function yields a value normalized on and additionally implements a threshold, where any value below that threshold is considered inadequate and requires adaptation to resolve. In some cases (e.g., Goal (A), (G), (I), and (J)), the utility function is trivially true in that it always is . Values such as these are generally not useful as points of adaptation given that there is no change, whereas Goal (D) will change drastically over the course of the game execution.
This section describes how self-adaptation can be applied to a gameplay loop via our motivating example and further presents initial experimental results gathered during gameplay by the authors.
We first developed a goal model of feesh as seen in Figure 2. For each goal we developed a utility function (e.g., Equations 1 and 2 for examples) to measure the satisfaction of each goal at run time. Additionally, we introduced adaptation mechanisms into feesh comprising software sensors (Monitoring) and logic for enabling self-reconfiguration (Analysis, Planning, Execution, Knowledge). Table 1 provides the adaptations currently available in feesh, however we anticipate that additional adaptation mechanisms can be introduced in future work. Each of these activities requires domain knowledge of the developer to (1) understand the underlying game logic/engine and know when/where software can be augmented with self-adaptive characteristics and (2) be able to derive new software artifacts and/or leverage existing software artifacts for use as first-class citizens at run time.
3.2. Adaptive Gameplay
We now present a sample use case for the MAPE-K loop within the context of the game. For example, consider Goal (C) in Figure 2. The utility function for this goal can be inserted into the main gameplay loop as part of the Monitoring phase. In the event that this goal is considered violated (e.g., ) then the adaptation mechanism will determine which reconfiguration is necessary for Goal (C) to be satisfied again (note: such goals are considered non-invariant in that they can tolerate temporary failure; an invariant goal cannot tolerate failure and as such the program would need to go to a safety state - e.g., shutdown or forcing user input). Depending on the complexity of the Analysis and Planning phases, the SAS may generate one or many possible reconfiguration strategies and select the most appropriate course of action. In the case of Goal (C), the system reduces by (where this value was determined empirically to extend gameplay). Moreover, if the exceeds the width of the canvas (i.e., nothing else is visible other than the player) then the game is considered to be won and the player is returned to the main menu.666With MAPE-K active, the won state is not attainable unless if the player gains mass quicker than can be processed. Figures 2(a) and 2(b) illustrate this adaptation in-game.
3.3. Initial Experimental Results
This section presents initial results on applying the MAPE-K feedback loop to feesh. For the purposes of this experiment, all utility values are normalized between and execution time relies on the frameCount variable provided by p5.js (i.e., integer ticks since program start). feesh was run on a GitHub.io-hosted page (https://efredericks.github.io/feesh/) and accessed via Google Chrome (Version 97.0.4692.71 (Official Build) (64-bit)) on a modern laptop. We performed two experimental treatments: MAPE-K enabled and MAPE-K disabled (i.e., Normal). For each treatment we performed replicates to establish statistical significance and we use the Wilcoxon-Mann-Whitney u-test with a significance (i.e., p-value) threshold of . The results were gathered by the authors of this paper. Additionally, we focus on Goal (F) to demonstrate feasibility of this approach, where we will explore additional metrics in future work.
Figure 3(a) presents boxplots of the overall execution time (i.e., ticks) for each treatment. As can be seen by these plots, players are engaged significantly longer by the version with MAPE-K enabled ( ), suggesting that the system is reconfiguring itself in support of its goals. Figure 3(b) presents the average utility values for Goal (F), where its utility equation is defined as:
, where is short for and is short for the canvas width. As can be seen in Figure 3(b), the average utility values for Goal (F) are significantly higher than those without MAPE-K ( ), suggesting that adaptations can continuously improve the satisficement of Goal (F). For the Normal system, the utility values remain high until the player wins the game (i.e., ).
Initially we had anticipated that the FPS would be the most significant hindrance to game execution, given the additional overhead of the p5.js library with respect to graphics processing, the large amount of entities that can spawn, and lack of interaction with a graphics processor (i.e., shaders are not compiled for this application). The game ran at approximately FPS with or without MAPE-K enabled (upon visual inspection), where the main limiting factor for goal satisfaction was . However, we did note that being overzealous with adaptations can in fact induce a performance decrease. For example, one reconfiguration strategy was in support of Goal (E) (i.e., Maximize Alive Enemies). A loop was added to scale the number of active enemies as appropriate to the player’s size and score but as a result the FPS significantly dropped and would cause violations to Goal (B) (i.e., [Maintain] 30 FPS), resulting in a failed execution as Goal (B) is invariant.
This paper has presented our approach for incorporating the MAPE-K feedback loop into the core gameplay loop of a single-player game. We developed a goal model and associated utility functions for a browser-based game centered around creative coding concepts. We additionally deployed a MAPE-K feedback loop to enable self-adaptation at run time in response to player actions and game mechanics. Initial results indicate that applying a self-adaptive overlay to a gameplay loop can enable adaptation at run time as a result of software engineering artifacts (i.e., by elevating artifacts to be first-class citizens in the core loop).
Future paths of research for this project include human-focused studies to determine the effectiveness of self-adaptation in games (including player skill as an adaptation metric), incorporation of run-time search techniques for optimization and testing, and expansion of the feesh game engine for both teaching and research.