Massively multiplayer online role-playing games (MMORPGs) attract hundreds of thousands of daily players. In MMORPGs, players control a virtual character, and are able to move around a vast, open world with different activities and in-game items. Like other online games, social interaction and skill advancement are central tenets in MMORPGs (Castronova, 2001). One crucial difference between MMORPGs and other online game genres is that each MMORPG typically maintains a robust in-game economy (Lehdonvirta, 2005b). In these economies, there exist thousands of items, many of which are exchanged for in-game currency. Thus, the many goods are dynamically priced according to the laws of supply and demand, rather than explicitly set by game administrators. Therefore, virtual economies display some similarities to real-world economies (Castronova, 2001).
Like real-world economies, MMORPGs witness economic phenomena such as inflation, illicit activity, and price manipulation (Castronova, 2001). In MMORPGs, these phenomena are especially important due to their direct impacts on player experience. Game developers play a large role in managing in-game economies by implementing policy changes, in our case a market intervention. For example, consider an item traded freely on the open market in a game. If a game developer institutes a game update where a player is now available to buy the item from a non-player character (NPC) shop, then they have, in effect, instituted a price floor for the aforementioned item. Game developers can also influence game economies by signaling updates through community-focused communications, as well. This is similar to how a central bank may signal interest rate changes (Johnson, 2002). Increasingly, MMORPGs such as Eve Online (Hillis, 2007) and Old School RuneScape (Jagex, 2022) have turned to economists to assess and manage their economies in recent years.
Inflation is an important economic phenomenon, with increasing relevance to both in-game and real-world scenarios. In MMORPGs, it is common to see both inflation or deflation (Castronova, 2001; Castronova et al., 2009; Castronova, 2008; Jung et al., 2011). The former is often times caused by a lack of “gold sinks”, which are in-game mechanisms to remove currency from circulation. These sinks can take various forms, such as through taxes or fees. Inflation may be further exacerbated by increasing numbers of high-level players who produce large amounts of gold or by illicit currency sellers who use massive bot farms to produce gold. On the other hand, deflation can become a problem when a game does not provide adequate currency-generating activities (Wright, 2021; Colbert, 2021). In such an instance, players may resort to bartering as opposed to using currency as an exchange mechanism. Nonetheless, both inflationary and deflationary environments have the potential to erode player experience, and are thus crucial to manage (Harambam et al., 2011).
MMORPG economies offer a unique window to study economic policy due to their frequent updates and markets that resemble the real-world (Castronova, 2003). In this paper, we explore the results of a natural experiment in Old School RuneScape (OSRS). Specifically, we analyze the effects of two economic interventions, a transaction tax and an item sink, in OSRS. We employ various quasi-experimental approaches, such as difference-in-difference, regression-discontinuity and regression-kink designs. Furthermore, we discuss the implications of the tax on real-world trading. We find that the item sink did increase the prices of the targeted items. On the other hand, trading volume was relatively unaffected by both the transaction tax and item sink. Furthermore, we observe that real-world trading prices were relatively unaffected by the market interventions. Our findings are not only useful to understanding and managing virtual economies, but also may provide a basis for understanding real-world economic phenomena through natural experiments found in many virtual economies.
The rest of the paper is structured as follows. In Section 2, we review the related literature regarding MMORPGs and virtual economies. Section 3 details RuneScape and its data. Furthermore, we analyze the dynamics of the RuneScape economy. In Section 4.1, we outline the natural experiment, our quasi-experimental method, and analyze the impacts of the transaction tax. Section 5 provides further discussion on our analysis, provides recommendations for natural experiments in MMORPG economies, and discusses limitations and future opportunities. Furthermore, we outline how natural experiments in virtual economies may be useful for game developers. Finally, we conclude the work in Section 6.
2. Related Work
2.1. Virtual Economies
Virtual economies, as a research domain, largely gained prominence due to MMORPGs in the early 2000s (Castronova, 2001). There exist many clear analogues to real-world economic phenomena in virtual economies. Although there remain some differences, Castronova asserts that virtual economies may still provide a ripe environment for understanding real-world economies (Castronova, 2003). Many of the same quantities studied in economics may also be calculated and studied in virtual economies. For example, Castronova provides a macroeconomic summary of EverQuest (Castronova, 2001). In their summary, they detail topics such as exchange rates between real-world and virtual currency, per-capita production, wages, inequality, and inflation. For EverQuest 2, Castronova et al. observe that aggregate economic behavior follows closely with that of the real-world; however, fluctuations are more dramatic (Castronova et al., 2009). In particular, they find high rates of inflation in some virtual economies.
Inflation is a core economic concept that concerns the increase of the price of goods and services. In online games, there exists a similar concept of mudflation (Castronova, 2008). Mudflation, an amalgamation of multi-user dungeon (MUD, a precursor genre to MMORPGs) and inflation, is a phenomenon in MMORPGs where prices of once expensive goods decline in value. Harambam et al. note that concerns over mudflation are noted not only by researchers but also by players themselves (Harambam et al., 2011). Price swings have significant effects on player retention. For example, Jung et al. note that deflation may cause boredom among players (Jung et al., 2011). On the other hand, inflation can significantly degrade player experience by making items unattainable, thus exacerbating inequality in a game. Inflation in a virtual game economy is largely determined by in-game economic mechanisms that regulate the production and consumption of currency (He et al., 2018). Lahti argues that if a player earns the most from trading, rather than generating new wealth, then inflation should stay low. They further suggest that the economy must be structured such that it utilizes low value items, regardless of player tenure (Lahti, 2015). Lehdonvirta and Castronova suggest that a monopsony market structure, where the game publisher buys goods from players, addresses the issue of deflation (Lehdonvirta and Castronova, 2014). Furthermore, they note that trading allows players to play according to their competitive advantages, and thus allows a player to focus on aspects of the game they enjoy most. Virtual economy inflation may also be affected by characteristics of the player base. For example, Belaza et al. suggest that players from countries with high real-world inflation tend to buy more than sell in-game, thus mimicking real-world behavior (Belaza et al., 2020).
Like policymakers in the real-world, game operators may introduce changes to the game economy through game updates. Also akin to the real-world, these policy changes may deliberately, or indirectly, morph the economy of a game. For example, Castronova highlights the case of Everquest, where significant item deflation was observed in items after game expansions (Castronova, 2001). For RuneScape, Bilir suggests that, in light of a game update which eased item trading costs, the overall price index decreased (Bilir, 2009). Game publishers may also directly impose economic policy, such as through price floors or ceilings. Castronova proposes that price ceilings do not generate excess demand and that price floors do not create excess supply (Castronova, 2003). Castronova further suggests that it may useful to directly control the prices of some goods. In games like RuneScape, in-game mechanisms, such as skills, provide price floors for certain goods. Game developers may test the effects of different policies through A/B testing on dedicated servers (Castronova et al., 2015)
. Stephens et al. propose a reinforcement learning approach to simulate the game economy to test the effects of updates on inflation(Stephens and Exton, 2021).
We build on prior work in the following directions. First, we utilize the market intervention aspect of game updates in a virtual economy. Specifically, we concern our analysis with the impact of a transaction tax and an item sink in Old School RuneScape, a popular MMORPG with tens of thousands of players. Furthermore, we employ multiple econometric methods to parse the independent causal effects of multiple market interventions, including difference-in-differences, regression-discontinuity and -kink designs. Lastly, we assess the effectiveness of the policy changes on item prices and trade volume relative to the policy goals of the game publisher.
2.2. Social Behavior in MMORPGs
Many MMORPGs have a high degree of player interaction, such as through text chat or trading mechanisms. These interactions have important effects on player retention and economic activity. For example, Jeong et al. find that friendship and trading interaction networks have high degrees of overlap compared to other types of social interactions (Jeong et al., 2015). In Aion, a popular MMORPG, Chun et al. observe that interactions such as message exchange are highly correlated with trade (Chun et al., 2018). Kang et al. show that people tend to join guilds, which are in-game groupings of players, when they expect benefits, such as money or valuable items (Kang et al., 2015). Furthermore, users often exhibit a profit-seeking mentality by switching guilds. Bisberg et al. find that players who experience or observe generosity in MMORPGs generally show higher future game engagement (Bisberg et al., 2022). Chung et al. observe that when a player group is not cohesive, there tends to be significant attrition (Chung et al., 2014). On the other hand, players who gather large amounts of currency or valuable items are less likely to churn from the game (Park et al., 2017).
2.3. Real-World Trading
Real-world trading (RWT) is an activity, often against most online games’ terms of service, where a player trades real-world currency for an in-game currency, item or service. The trade between real-world currency and in-game currency has existed since MMORPGs were first popularized (Castronova, 2001). Oftentimes, large bot farms are used to generate currency for RWT (Lee et al., 2016). Wohn et al. find that large friend networks, along with high rates of virtual good exchange, were positively associated with a user engaging in RWT (Wohn, 2014). Furthermore, they find that users who engaged in high amounts of RWT were primarily buying items for their visual effects, whereas low spenders were buying consumable items for playing the game. Similarly, Xu et al. find that server population and network density are correlated with in-game currency price (Xu et al., 2017).
The effect of RWT on MMORPGs is disputed. Lehdonvirta suggests that some subsets of players may see RWT as beneficial, such as achievement and immersion-oriented players (Lehdonvirta, 2005a, b). Likewise, Jung et al. suggests that RWT is beneficial to game operators and that there exists an optimal amount of supply for game items (Jung et al., 2011). On the other hand, Huhh suggests that RWT may have an impact on player retention, and points to the demise of Lineague 2 as an example of RWT’s effect on player experience (Huhh, 2005).
The size of RWT in a game can be large. For example, Chun et al. show that over half of trades involving upper-class Aion players are associated with RWT entities (Chun et al., 2018). Due to the perceived harm that botting, the activity of using an automated program to play the game, may have on the game, many approaches have been adopted to identify bots and accounts involved in RWT. Fujita et al. exploit the trading network of a game to calculate the likelihood of users being involved in RWT activity (Fujita et al., 2011). Similarly, Jeong et al. find that social interaction networks varied substantially between bots and human players (Jeong et al., 2015). Lee et al. construct a trading network and decompose it into various communities (Lee et al., 2018)
. They find that professional RMT providers typically form specific network structures, such as star-shaped or chain networks. Finally, Tao et al. use machine learning to identify RWT associated accounts(Tao et al., 2019).
3. RuneScape Markets
Old School RuneScape (OSRS) is the latest rendition of the long-running RuneScape MMORPG series, which was first released in 2001. Functionally, OSRS retains many characteristics of the 2007-era version of RuneScape, remaining visually and mechanically similar, but has received regular updates. In this section we describe OSRS, its in-game market, and its data.
3.1. Game Description
OSRS, which typically has at least 50,000 accounts logged in at any given time, is an MMORPG set in a large and open fantasy world. In this world, players can gain experience towards a variety of in-game skills and interact with other players. Skills are generally broken down into four categories: (1) combat skills, (2) gathering skills, (3) artisan skills, and (4) support skills. While progressing most skills requires some form of item consumption, skills may also produce many items as a byproduct of their progression. These items are often exchanged for gold pieces (“GP”) with in-game shops, through NPCs, or with other players. For example, training one’s “woodcutting” skill produces logs. These logs may be purchased by players who wish to train their “fletching” or “firemaking” skills. Thus, there is a significant amount of player interaction in OSRS, especially regarding the exchange of items. Although OSRS is free-to-play, many players opt to buy monthly membership, which allows for much broader content, including new areas, items, and skills.
3.2. The Grand Exchange
The Grand Exchange (“GE”) is a marketplace that connects buyers and sellers of all OSRS tradeable items, similar to a real-world securities exchange. The GE works by clearing offers between buyers and sellers in the order it receives them. For example, suppose a buyer places an order for an item at and there exists a sell offer at price . Then, the buyer receives the item at price and also receive in gold; the seller receives . If a seller places an order at price and a buy offer exists at price , then the seller receives gold in exchange for the item. Thus item prices are determined by the market, and vary over time. Items also have a buy limit, which restricts the quantity of an item that a player can purchase within a four hour game window. There is no sell limit. We show an example of the GE interface in Figure 1. In total, a player who pays for a RuneScape membership may have up to eight concurrent buy and sell orders.
OSRS contains roughly four thousand items that are able to be traded on the GE. Jagex, OSRS’s publisher, provides prices for these items on the official game website, but these prices are generally delayed, sometimes by as much as 24 hours. Conversely, the OSRS Wiki, the official community wiki, provides an endpoint which returns the average instant-buy and instant-sell prices and volumes for a given item. To do so, they leverage a partnership with RuneLite, a popular, open-source third-party client used to play OSRS. RuneLite is officially recognized by Jagex as one of three approved third-party clients. At the time of writing, roughly 75% of the player population used RuneLite. Whenever a player using RuneLite bought or sold an item on the GE, we collected the average daily buy and sell prices and trade volumes for all items from August 2021 to August 2022.
3.4. RuneScape Economy
As noted by Lehdonvirta and Castronova, players oftentimes play towards their own competitive advantages, and thus are likely unable to produce all items that meet their consumption needs (Lehdonvirta and Castronova, 2014). Thus, the OSRS economy has notable similarities to real-world economies: players work towards producing items or achieving goals and they trade for products they want or need. The GE is the primary mechanism where players sell what they produce, and trading on the GE represents the majority of economic activity in OSRS (Bilir, 2009).
Our data cover roughly four thousand items in the OSRS universe; however, only about 70% of these items are traded on a daily basis. Concerning this subset of items, we calculate the correlation in price movements between all items. Specifically, we find that, on average, an item has a correlation with roughly 200 items. When we change the cutoff to , the number of correlated items rises to around 1,100. Thus, we find that many items exhibit strong price correlation with other items. This is expected, as many items are purchased jointly. For example, consider the item “Dragon Crossbow,” a popular in-game weapon. The Dragon Crossbow has a correlation of with the item “Armadyl d’hide body,” an armor piece that gives a bonus to using the Dragon Crossbow. Thus, the two goods are complementary, and therefore one would expect a high degree of (positive) price correlation between the two.
In aggregate, our data indicate an average of 4.5 trillion GP worth of items exchanged over the exchange, daily. The bond, the sole in-game item that a player may purchase with real-world currency, can also be bought and sold on the Grand Exchange for GP. Bonds may be used in-game to purchase “membership,” which unlocks additional content. If we assume an average GP price of $1.45, which is roughly the price as determined by a bond, then the implied daily volume of trade is $6.5 million. The implied yearly volume is thus $2.3 billion, which is larger than the gross domestic product of some countries. Furthermore, it is also important to note that our data are only reflective of the orders made by 75% of accounts, and thus our estimate is likely an underestimate of the true daily transaction volume in OSRS.
We can measure classic macroeconomic phenomena using the OSRS GE data. A price index tracks the price of a basket of goods, relative to a point in time. In the real-world, governments calculate price indices to track price inflation. Using our GE data, we construct a price index by calculating the mean price of goods (weighted by trade volume) in each week, and compare each week’s percent change in price relative to the week preceding new market interventions. To construct the price index, consider for an item ’s sale price in week , its trading volume, and the base week to compare to. Then, we construct the price index for a group of items in week by comparing the weighted average prices , as in equation (1
). In practice, we perform this calculation via linear regression, separately for three item groups, as shown inFigure 2.
Item prices were, for all items, rising for the final months of 2021, yet the first few months of 2022 saw the average price fall of 10 percentage points, and relatively stable prices in the middle of 2022, changing no more than 5% in aggregate from April 2022 to August 2022. On the other hand, items which were affected by a new trading sink, which we describe in depth in Section 4.2, experienced falling prices prior to the market intervention. Afterwards, these items stabilized and gradually increased post-intervention. In Section 4, we discuss and analyze the market interventions in detail.
4. Analyzing Market Interventions
Jagex announced two large market interventions to trading on the GE in November 2021: a transaction tax and an item sink (Jagex, 2022). Both market interventions were formally introduced in December 2021, and have stayed active to date. Jagex had publicly expressed concern that high-level items, items limited to players with a lot of game experience (and GP), were too common-place. And so the company implemented a two-pronged market intervention to tackle the issue of item pricing. First, the transaction tax collects 1% of the item sell price (subject to restrictions) for every GE transaction, thus impacting the profitability of real-world traders and botters who use the GE at high volume to generate GP. The tax is taken out of the seller’s proceeds. Second, the proposed item sink uses the proceeds from the transaction tax to remove high-level items by directly buying the items from players on the GE. If the virtual coffers filled by the transaction tax are insufficient, the item is not removed. The number of items to be removed and deleted is subject to a daily maximum set by Jagex. In this section, we use quasi-experimental techniques to analyze the effects of the tax on item prices and trading volumes. We do so by exploiting discontinuities in the tax rates, comparing sinked items to a plausible control group, and inferring trends in real-world trading via the price of illict in-game currency purchases.
The combined policy change is largely similar to real-world market interventions. A financial transaction tax is a popular economic policy that regulators employ to discourage market volatility and speculation, by increasing the price of an additional trade, and thus reducing trade volume. Many countries levy a financial transaction tax in their securities exchanges, and the European Union recently considered a Union-wide financial transaction tax (Rodríguez, 2021). The proposed item sink mirrors government efforts to remove items from circulation via market mechanisms, such as the Australian government’s fire-arms buyback in 1996 (Brew, 2017). Lastly, the tax and sink are redistributive in nature, since proceeds of the transaction tax are then passed back to the public (in this case, the player base) via another program. This is in line with the economic policy approach of tying a tax to another policy provision (Golladay and Haveman, 1977), such as ear-marking proceeds from gambling taxes to fund education (Simmons and others, 2006).
4.1. Transaction Tax Effects
We analyze the causal effect of the transaction tax on item price and trade volume by exploiting the non-linear jumps in the tax rate. The tax is 1% of the item sale price, but only applies if the price is greater than 100 GP. The tax is capped at 5 million GP regardless of item sell price. This leads to two distinct discontinuities: (1) a jump in the tax rate of sales above and below 100 GP, and (2) a change in the slope of the tax rate above and below 500 million GP. These first-stage discontinuities are exhibited in Figure 3. We use this approach to identify the effect of the transaction tax; the method produces point estimates for the causal effect at the tax cut-off points (Hahn et al., 2001). Notably, provision of the tax is decided by an item’s price, so we cannot disentangle the causal effect of the tax on item price, and thus we focus on the effect on trading volume. Alternative methods, such as a correlation analysis, would only be able to show the association between the policy change and economic outcomes, but our approach allows for the analysis of the causal relationship. Unless otherwise noted, our analysis considers the outcomes with a log transformation. The log transformation is monotonic, so does not meaningfully affect results, and allows for a percent-change interpretation of the estimated effects.
There is a sharp-discontinuity in tax rate at 100 GP, so we estimate the average effect of the 1% transaction (on items with sale price of 100 GP) through a regression discontinuity (RD) design (Hahn et al., 2001). This approach assumes that items are virtually the same, apart from the tax applied at the cut-off, in a neighborhood around the 100 GP cut-off. Around 100 GP, the effect of the transaction tax on trading volume is not distinguishable from zero at any standard level of significance. Figure A1 shows the graphical representation of the results, where the estimated discontinuity in volume above and below 100 GP is not distinguishable from zero.
Secondly, there is a discontinuity in the rate of change of the tax rate at 500 million GP, so we estimate the average effect of the 1% transaction (on items with sale price of 500 million GP) through a regression kink (RK) design (Card et al., 2017). This approach again assumes that items are virtually the same apart from a differing tax rate, in a neighborhood around the 500 million GP cut-off. Around 500 million GP, the transaction tax of 1% reduced trading volume by 6.9%, yet bounds on the treatment effect range -13 to -1% at a 95% significance level. This means that, while the reduction on trading volume is accepted as non-zero, our exact estimate is not precise, and could reasonably be as large as -13%, or as small as -1%.
4.2. Item Sink Effects
We analyze the causal effect of the new item sink on item price and trading volume by comparing the items that the new sink applied to against a plausible control group of items. The difference-in-difference (DiD) method (Card and Krueger, 1994; Bertrand et al., 2004), popular in empirical economics, generalizes the insight of John Snow’s 1856 cholera study (Snow, 1856) by comparing the difference in treatment effect between treatment and control groups before and after a new policy. DiD allows for a causal comparison in outcomes between a treatment group and a control group when researcher-directed randomization did not occur, as is the case of the item sink. In our case, the treatment group is the items affected by the sink, and we address concerns for treatment selection by constructing a control group of items which the sink could have reasonably applied to. Section A.3 provides further technical details of the DiD method.
Jagex implemented the item sink in two rounds. The first round on 09 December 2021 started buying and removing 34 items, all of which are high-level items which only very experienced players can afford and use. The second round started additionally buying 27 more high-level items on 16 December 2021. We treat each round as a separate policy event by comparing the items in the first round and second round separately to a control group that we construct. Given a set of all high-level items , all of which have an average price greater than GP, and the set of sink-targeted items , we define the control set
where measures the price correlation between items and . The construction infers which items show no significant price correlation (positive or negative) with the sinked items, and thus are neither reasonable substitutes nor complements to the sinked items. By construction, this control set represents similarly high-priced items to those in the sink, yet changes in demand motivated by the new item sink should not otherwise affect demand for items in the control group.
To verify our approach, we investigate the time trend in price before and after the market intervention, for both and in Figure 4
. Prior to the first sink on 09 December 2021, the sinked and control items do not have trends distinguishable from each other, and the control group is not meaningfully affected by the new item sink. Although the trend for control group prices appear positive, the trend is small in magnitude and thus the associated standard errors of the point estimates consistently contain zero, and the trend is not statistically distinguishable from no-trend. The lack of distinguishable trend between the groups validates the assumption that pre-trends are similar, thus justifying the DiD approach we employ to identify the causal effect of the sink on item prices. By the same argument, we validate the pre-trends assumption for trading volume inFigure A3.
The first treatment group experiences a jump in prices on the order of 7% when the sink was implemented, with a confidence interval of 5.0% to 9.6% at the 95% confidence level. This corresponds to the jump in prices for the sinked group on 09 December 2021, minus the change for the control items, in Figure 4 (and we present full point estimates in the appendix). We also estimate that items in the second sink experienced a jump in prices on order of 14.4%, with a confidence interval of 11.9% to 16.9% at the 95% confidence level. On the other hand, the new item sink’s effect on items’ trading volume was indistinguishable from zero in both the initial and second round of the sink. We show the visual representation of the DiD estimates for price in Figure 5, and for volume in Figure A4.
4.3. Real-World Trading
The last topic of analysis is real-world trading, where we infer interactions between real-world players and the virtual economy via different exchange rates for virtual currency. Players can buy “bonds” from Jagex, and these bonds are in-game items that grant the player a 14-day game membership. Importantly, bonds may also be sold between players on the GE. Thus, although the bond’s price is fixed in the real-world ($6.99 for our analysis period), the in-game GP price of the bond floats on the GE via an auction mechanism. This gives us a measure of the varying price (in US dollars) for in-game currency (GP), from which we can infer if there have been any changes in demand from players. Notably, buying bonds and selling them in-game for GP is explicitly allowed by Jagex’s terms of service and does not fall under the realm of real-world trading.
On the other hand, players can buy in-game GP from real-world traders illicitly (i.e., breaking the OSRS terms of service). Real-world traders exploit a number of methods to generate in-game currency in a cost-effective manner, from utilizing sophisticated procedures to automate in-game production (i.e. “botting”) to employing operators to manually play different accounts. Real-world traders typically generate huge sums of in-game GP by producing large numbers items, and then selling these items to other players on the GE for large sums of GP. In doing so, real-world traders generate large numbers of items in a manner that is explicitly banned by Jagex. In relation to the items which are affected by the item sink, real-world traders increase the supply of items that Jagex has designed to be in limited supply, and thus decrease the market price for items that Jagex has designed to be very expensive.
These activities are illicit as they are explicitly banned by the game operator, Jagex, who has has been tackling real-world traders and botters for at least the last decade (Guest, 2010). Jagex is active in enforcing their terms and services and they often ban accounts for which they find evidence of botting or real-world trading activity, confiscating any items generated in the process. Real-world traders sell their produced GP to players via illicit external websites; this is an illicit black-market, where scams and risk of a game ban is high. Because of these market frictions, when compared to the official bond market, there is a risk premium for illicit GP purchases, measured by the difference in illicit and official price for GP (in USD). As such, we collected the US dollar price-per-million GP from seven popular illicit gold sellers over our sample period, and compared their mean price to that in the official bond market.
We see that the official price-per-million GP varies between $2.50 and $6.50, and exhibits no explicit long-run increasing or decreasing trend. The official price is centered around a mean of $4.17. The illicit market has virtually no variation to speak of: a million GP sells on the black market at an average of around $0.495 for our entire sample period, and all sellers’ prices are between $0.45 and $0.58 – a very tight spread compared to the official channel. While the ten illicit sellers’ prices varied between 40 and 60¢for a million GP across the entire time period, prices on both sides of the new market interventions reach both ends of the range before and after
the market interventions. We tested for structural breaks in both time series, and found no statistically significant breaks in mean or variance.
Notably, over this time period, supply of GP in the official bond market saw no explicit changes. Jagex did not change any policies for bond buying between September 2021 and May 2022, and most variation in the official price will have arisen from changes in player demand. We do see short-run increases in the US dollar price for GP around both the market intervention announcements and the implementation. But, the short-run increases are not particularly large, and are overshadowed by larger short-run changes in different months (where there was no other market intervention).
Although not a stated target of the market interventions, real-world traders and botters are affected by the market interventions due to an increased cost of operation through the GE transaction tax. Every time a real-world trader sells items on the GE, they pay a transaction tax of 1%, and thus face higher costs compared to the time before the transaction tax existed. And yet, illicit real-world traders did not change the price they charged for in-game GP around the market interventions. As such, we infer that real-world trading, economic interactions between the real-world and the in-game economy of OSRS, were not meaningfully affected by the market interventions.
In this study we applied methods from empirical economics to estimate the causal effects of new market interventions in a large-scale virtual economy. We found that the the interventions did not uniformly affect economic outcomes: prices for high-level items increased due to the item sink, but trading volume was not meaningfully affected by the transaction tax. Our results align clearly with standard economic theory: the new item sink increased demand for high-level items (by buying items), thus leading to increased prices. And yet, gauging whether the market interventions were successful depends on the policy objective of the game planner.
Policy Objectives. Jagex has been monitoring and stabilizing its in-game economy since the early 2000s, and has been combating real-world trading since before then. The 09 December 2021 market interventions were a very consequential change to the OSRS economy, but Jagex releases other game updates and additions regularly. While it is clear that the transaction tax meaningfully reduces the amount of in-game currency in circulation, and the item sink reduces the number of (targeted) high-level items in circulation, Jagex has not publicly stated their intentions for this round of market interventions. Without a definite policy target, it is not possible to state whether the game designer achieved their policy goal with these market interventions. Thus, we leave it for future work to infer policy goals in virtual market interventions, and whether policy changes achieved these goals.
Limitations. Our study estimates causal effects of two main policy interventions on economic and trading activity in the OSRS virtual economy. We conduct our analysis with data on item trading, over a number of months around new market interventions. As such, we are able to estimate causal effects on trading prices and volume at the item-level, in the short to medium time-run. As we do not have data on player-level interactions, we cannot further analyze the effects on player behavior as a whole. Furthermore, we only observe prices in the real-world trading market (for virtual currency), and cannot analyze the prevalence of real-world trading directly. Lastly, our data cover the majority of OSRS trading data, though not the whole market, as it is sourced in partnership with RuneLite.
While we do not use player-level data, our analysis plausibly identifies causal estimates for the effect of market interventions in the OSRS economy. Uniquely, we did so for interventions that were implemented in the past, without our input on their design. Thus, we performed no researcher-guided randomization or A/B design in our causal study. Instead, we leveraged context-specific information regarding the design of the market intervention, combined with methodology common in empirical economics. Firms that operate virtual (or even real-world) marketplaces routinely partner with researchers to study the effects of the prospective policy change, but these arrangements are not without costs. Researchers must rely on firms to access their data and marketplace, researcher-guided randomization of new policies are often very costly to implement, and researchers must clear ethical guidelines when giving a new policy change to customers operating in such marketplaces. While our approach has a number of limitations, our approach has the benefit of avoiding these costs, while maintaining a causal interpretation. Thus, we did not deal with such problems or costs common for similar, causal analyses of market interventions. To our knowledge, this is the first analysis to apply such methods to the setting of a virtual economy; we set the stage for further work to utilize these methods for the study of virtual economies, overcoming problems related to traditional researcher-guided randomization and A/B testing.
Future Work. There are several avenues of future work as we believe our approach is generalizable across many games, since item exchanges are common in many online games. For example, World of Warcraft, a popular MMORPG, maintains an “auction house” that serves much of the same functions that the Grand Exchange offers. These online exchanges also may institute different policies. For example, in Guild War 2’s auction house, there exists a fee to list an item for sale (5% of sale price), and the exchange takes a 10% portion of every sale. In both of the former games, the exchanges are global to the server. EVE Online, a space-themed MMORPG, maintains distributed marketplaces in the various “star systems”. The EVE Online market also has various fees, such as a broker’s fee, a sales fee or a relisting fee. Changes in any of the aforementioned fees may serve as unique natural experiments to perform studies like ours. To acquire the data necessary for these macro-level studies, one may consult official game APIs or those run by third-party community operators.
Our approach need not only be applied to MMORPG economies, but may also be generally applied to online marketplaces. Many online marketplaces routinely update their platforms to institute different fees. For example, consider eBay, an online marketplace which charges insertion fees (for listing creation) and final value fees (for an item sale). These fees vary by item and by final sale value. On October 10, 2022, eBay will change their sneakers sales fees, specifically for sneakers sold between $100 and $150 (Team, 2022). Accordingly, like in our analysis, there exist many discontinuities in the way a marketplace like eBay structures its fees that may be worthy of analysis. Another type of marketplace to consider are cryptocurrency markets, which also demonstrate rapidly changing fees and rules due to dynamic regulatory environments (Shanaev et al., 2020). Given the increasing levels of ecommerce and cryptocurrency activity, it will be important to understand how such markets react according to changing policies. Increasingly so, regulatory agencies, such as the Security and Exchange Commission have looked to MMORPG economies as units of study (Down, 2021). Virtual worlds have also been noted for their potential to study other concepts such as in epidemiology (Lofgren and Fefferman, 2007) or social networks (Bainbridge, 2007).
This work analyzes the impacts of various market interventions in a large virtual economy. Specifically, this study concerns a transaction tax and an item sink introduced into the main item exchange in OSRS, a popular MMORPG. Utilizing quasi-experimental techniques commonly used in empirical economics, we find that while the market interventions did lead to inflation in the price of the targeted items via the item sink, the trading volume of these items was not meaningfully affected. Furthermore, we also note that the real-world trading market remained relatively unaffected by the market interventions. Finally, we also discuss the parallels between real-world economic phenomena and those witnessed in large-scale game economy such as OSRS. Our approach reinforces the idea that virtual economies and marketplaces may also be a useful tool in analyzing real-world economic theory. Since game designers and marketplace operators routinely introduce policy updates, we see much potential for future studies, like ours, that study economic phenomena through large virtual economies and quasi-experimental methods.
Acknowledgements.This collaboration has been supported by a grant from Capital One. Silva’s research has also been supported by NASA; NSF awards CNS-1229185, CCF1533564, CNS-1544753, CNS-1730396, CNS-1828576, CNS-1626098; and DARPA PTG and D3M. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA, NSF, NASA, or Capital One. The authors would like to thank the maintainers of the OSRS Wiki for maintaining GE API endpoints and documentation, as well as providing crucial guidance.
- The scientific research potential of virtual worlds. Science 317 (5837), pp. 472–476. External Links: Cited by: §5.
- On the connection between real-world circumstances and online player behaviour: the case of eve online. PloS one 15 (10), pp. e0240196. Cited by: §2.1.
- How much should we trust differences-in-differences estimates?. The Quarterly journal of economics 119 (1), pp. 249–275. External Links: Cited by: §A.3, §4.2.
- Real economics in virtual worlds: a massively multiplayer online game case study: runescape. Available at SSRN 1655084. Cited by: §2.1, §3.4.
- The gift that keeps on giving: generosity is contagious in multiplayer online games. CoRR abs/2207.10615. External Links: Cited by: §2.2.
- National firearms amnesty. Parliament of Australia. External Links: Cited by: §4.
- Optimal data-driven regression discontinuity plots. Journal of the American Statistical Association 110 (512), pp. 1753–1769. External Links: Cited by: §A.1, §A.2.
- rdrobust: An R Package for Robust Nonparametric Inference in Regression-Discontinuity Designs. The R Journal 7 (1), pp. 38–51. External Links: Cited by: §A.1, §A.2.
- Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania. American Economic Review 84 (4), pp. 772–793. External Links: Cited by: §A.3, §4.2.
- Regression kink design: theory and practice. In Regression discontinuity designs, External Links: Cited by: §A.2, §4.1.
- Policy questions raised by virtual economies. Telecommunications Policy 39 (9), pp. 787–795. Cited by: §2.1.
- As real as real? macroeconomic behavior in a large-scale virtual world. New Media Soc. 11 (5), pp. 685–707. External Links: Cited by: §1, §2.1.
- Virtual worlds: a first-hand account of market and society on the cyberian frontier. Available at SSRN 294828. Cited by: §1, §1, §1, §2.1, §2.1, §2.3.
- On virtual economies. Game Stud. 3 (2). External Links: Cited by: §1, §2.1, §2.1.
- Synthetic worlds. In Synthetic Worlds, Cited by: §1, §2.1.
- Unveiling a socio-economic system in a virtual world: A case study of an MMORPG. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, P. Champin, F. Gandon, M. Lalmas, and P. G. Ipeirotis (Eds.), pp. 1929–1938. External Links: Cited by: §2.2, §2.3.
- Unveiling group characteristics in online social games: a socio-economic analysis. In 23rd International World Wide Web Conference, WWW ’14, Seoul, Republic of Korea, April 7-11, 2014, C. Chung, A. Z. Broder, K. Shim, and T. Suel (Eds.), pp. 889–900. External Links: Cited by: §2.2.
- New world’s economy is so busted, players are bartering for items. Kotaku. External Links: Cited by: §1.
- Runescape’s economy is so cracked that the us government is watching it. External Links: Cited by: §5.
Detecting real money traders in MMORPG by using trading network.
Proceedings of the Seventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE 2011, October 10-14, 2011, Stanford, California, USA, V. Bulitko and M. O. Riedl (Eds.), External Links: Cited by: §2.3.
- The economic impacts of tax—transfer policy: regional and distributional effects. Elsevier. Cited by: §4.
- Jagex ltd. v. impulse software: an almost big win for online-gaming software developers. SMU Sci. & Tech. L. Rev. 14, pp. 497. Cited by: §4.3.
- Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica 69 (1), pp. 201–209. External Links: Cited by: §A.1, §4.1, §4.1.
- Game over? negotiating modern capitalism in virtual game worlds. European Journal of Cultural Studies 14 (3), pp. 299–319. Cited by: §1, §2.1.
- Study on the inflation in the economic system in online games. In 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), pp. 1–2. Cited by: §2.1.
- Virtual world hires real economist. Thomson Reuters. External Links: Cited by: §1.
- Empirical study on the decline of lineage 2 in korea. Available at SSRN 833847. Cited by: §2.3.
- Old school economy - future plans. Old School RuneScape. External Links: Cited by: §1, §4.
- Analysis of game bot’s behavioral characteristics in social interaction networks of MMORPG. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, SIGCOMM 2015, London, United Kingdom, August 17-21, 2015, S. Uhlig, O. Maennel, B. Karp, and J. Padhye (Eds.), pp. 99–100. External Links: Cited by: §2.2, §2.3.
- The effect of inflation targeting on the behavior of expected inflation: evidence from an 11 country panel. journal of monetary economics 49 (8), pp. 1521–1538. Cited by: §1.
- Analysis of the relationship between virtual goods trading and performance of virtual worlds. Available at SSRN 1938313. Cited by: §1, §2.1, §2.3.
- Rise and fall of online game groups: common findings on two different games. In Proceedings of the 24th International Conference on World Wide Web Companion, WWW 2015, Florence, Italy, May 18-22, 2015 - Companion Volume, A. Gangemi, S. Leonardi, and A. Panconesi (Eds.), pp. 1079–1084. External Links: Cited by: §2.2.
- Inflation control in virtual economies. Cited by: §2.1.
- No silk road for online gamers!: using social network analysis to unveil black markets in online games. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018, P. Champin, F. Gandon, M. Lalmas, and P. G. Ipeirotis (Eds.), pp. 1825–1834. External Links: Cited by: §2.3.
- You are a game bot!: uncovering game bots in mmorpgs via self-similarity in the wild. In 23rd Annual Network and Distributed System Security Symposium, NDSS 2016, San Diego, California, USA, February 21-24, 2016, External Links: Cited by: §2.3.
- Virtual economies: design and analysis. Mit Press. Cited by: §2.1, §3.4.
- Real-money trade of virtual assets: ten different user perceptions. Proceedings of Digital Arts and Culture (DAC 2005), IT University of Copenhagen, Denmark, pp. 52–58. Cited by: §2.3.
- Virtual economics: applying economics to the study of game worlds. In Proceedings of the 2005 Conference on Future Play (Future Play 2005), Lansing, MI, Cited by: §1, §2.3.
- The untapped potential of virtual game worlds to shed light on real world epidemics. The Lancet Infectious Diseases 7 (9), pp. 625–629. External Links: Cited by: §5.
- Achievement and friends: key factors of player retention vary across player levels in online multiplayer games. In Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, April 3-7, 2017, R. Barrett, R. Cummings, E. Agichtein, and E. Gabrilovich (Eds.), pp. 445–453. External Links: Cited by: §2.2.
- R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. External Links: Cited by: §A.1, §A.2.
- Financial transaction tax in europe. EC Tax Review 30 (4). External Links: Cited by: §4.
- Taming the blockchain beast? regulatory implications for the cryptocurrency market. Research in International Business and Finance 51, pp. 101080. Cited by: §5.
- Gambling in the golden state 1998 forward. Cited by: §4.
- On the mode of communication of cholera. Edinburgh medical journal 1 (7), pp. 668. Cited by: §4.2.
- Advances in measuring inflation within virtual economies using deep reinforcement learning. In Agents and Artificial Intelligence - 13th International Conference, ICAART 2021, Virtual Event, February 4-6, 2021, Revised Selected Papers, A. P. Rocha, L. Steels, and H. J. van den Herik (Eds.), Lecture Notes in Computer Science, Vol. 13251, pp. 294–314. External Links: Cited by: §2.1.
- MVAN: multi-view attention networks for real money trading detection in online games. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, A. Teredesai, V. Kumar, Y. Li, R. Rosales, E. Terzi, and G. Karypis (Eds.), pp. 2536–2546. External Links: Cited by: §2.3.
- Updates to the sneakers experience on ebay. eBay. External Links: Cited by: §5.
- Spending real money: purchasing patterns of virtual goods in an online social game. In CHI Conference on Human Factors in Computing Systems, CHI’14, Toronto, ON, Canada - April 26 - May 01, 2014, M. Jones, P. A. Palanque, A. Schmidt, and T. Grossman (Eds.), pp. 3359–3368. External Links: Cited by: §2.3.
- Amazon’s ’new world’ is suffering from a bizarre currency crisis. Input. External Links: Cited by: §1.
- Examining the effects of network externalities, density, and closure on in-game currency price in online games. Internet Res. 27 (4), pp. 924–941. External Links: Cited by: §2.3.
Appendix A Appendix
a.1. Regression Discontinuity
The Regression Discontinuity (RD) design (Hahn et al., 2001) is a method devised in econometrics to identify the causal effect of a treatment (often a change in policy), exploiting the fact that some policies are provided discontinuously according to a running variable. The crucial assumption is that outcomes are continuously identical around the cut-off point, and their only difference when converging to the cut-off point above and below is treatment status. In our case, we use the RD approach to analyse the effect of the new transaction tax on (log) trading volume. Specifically, the tax is 1% if price is above 100 GP (and below 500 million GP), and 0% if below.
We estimate equation (3), where refers to item ’s price on day , its trading volume, the indicator function, and the error term. counts the quantity of item traded on day , and price is measured in virtual currency (GP). represents an unknown nuisance function of how volume relates to item price, which is estimated separately above and below the cut-off point.
Parameter is the estimand, in this case the local average treatment effect (at ) of the 1% transaction tax on trading volume. In practice, we estimate a local polynomial above and below the cut-off point, and the discontinuity between the two. We restrict our data to observations within 20 GP of the cut-off (i.e. 80 to 120 GP), and for the month following the policy announcement, dates 09 December 2021 to 01 January 2022. We implement this procedure via rdrobust (Calonico et al., 2015b, a) for R (R Core Team, 2022).
The estimand is the distance between the two red lines in Figure A1; the red lines are the estimated local polynomial above and below the cut-off, and the dots are average trading volume for (binned) prices (and have associated weights which are not in the figure). In this case, the distance between the lines at
is not statistically distinguishable from zero, and so we accept the null hypothesis that the transaction tax did not affect items’ trading volume (around price of 100 GP).
a.2. Regression Kink
The Regression Kink (RD) design (Card et al., 2017) is a method devised in econometrics to identify the causal effect of a treatment (often a change in policy), exploiting the fact that some policies (mostly taxes) are provided at different rates (discontinuously) above and below according to a correlated variable. The crucial assumption is that outcomes are continuously identical around the cut-off point, and their only difference when converging to the cut-off point above and below is treatment rate. In our case, we use the RK approach to analyse the effect of the new transaction tax on (log) trading volume. Specifically, the tax is 1% if price is below 500 million GP (above 100 GP), and 5 million GP if above. This variation leads to a change in rate of tax above and below the 500 million GP cut-off (see Figure 3).
We estimate equation (4), where refers to item ’s price on day , its trading volume, the new transaction tax, and the error term. represents an unknown nuisance function of how volume relates to item price, which is estimated separately above and below the cut-off point.
Parameter is the estimand, in this case the local average treatment effect (at ) of the 1% transaction tax on trading volume. In practice, we estimate a local polynomial above and below the cut-off point, and the discontinuous difference between their slopes. We restrict our data to observations with price greater than 100 million GP, and for the month following the policy announcement, dates 09 December 2021 to 01 January 2022. We implement this procedure via rdrobust (Calonico et al., 2015b, a) for R (R Core Team, 2022).
The estimand is the slope differences between the two red lines in Figure A2 at the cut-off point; the red lines are the estimated local polynomial above and below the cut-off, and the dots are average trading volume for (binned) prices (and have associated weights which are not in the figure). In this case, the difference in slope between the lines at is statistically distinguishable from zero, and so we reject the null hypothesis that the transaction tax did not affect items’ trading volume (around price of 500 million GP).
a.3. Difference in Difference
The Difference in Difference (DiD) design (Card and Krueger, 1994; Bertrand et al., 2004) is a method devised in econometrics to identify the causal effect of a treatment, when there is a plausible control group who did not receive the treatment. The crucial assumption is that outcomes are the same for treatment and control groups follow parallel trends before the treatment was provided. In our case, we use the DiD approach to analyse the effect of the new item sink on items’ trading prices and trading volume.
For the RD model, we estimate Equation 5, where outcome is an item’s trading price or trading volume. is a binary for whether the observation is after the policy implementation date, 09 December 2021, a binary for whether the item is targeted by the new item sink. Notably, we include fixed effects at the item-level, , in order to average out systematic item differences in price and trading volume — and this term identifies treatment status, so that a binary for cannot be including separately.
Parameter is the estimand, in this case the average treatment effect on the treated of the item sink on price and trading volume. In practice, we estimate via linear regression, restricting to observations which either our in our constructed control group (2) or the item-sink targeted (separately for our tax). We only consider observations for after the policy announcement, 09 November 2019, and the month following the policy implementation, until 01 January 2022.
Our point estimate for the causal effect of the sink is presented in Table A1, on the row labelled “Post Treated.” The figures in brackets below each line are the associated standard errors of the point estimates, which are used to construct the point estimates in Figure 5 and Figure A4. In this case, the sink is estimated to have positively affected price of the items in the sink (in both rounds of implementation), but not their trading volume.