Addon to Cookie Clicker
We study the incremental knapsack problem, where one wishes to sequentia...
As item trading becomes more popular, users can change their game items ...
Modern board games are a rich source of interesting and new challenges f...
Understanding whether preferences are sensitive to the frame has been a ...
For sustainable growth and profitability, online game companies are
We introduce a new abstract graph game, Swap Planarity, where the goal i...
M-GWAP is a multimodal game with a purpose of that leverages on the wisd...
Addon to Cookie Clicker
Automatic Playthrough for Cookie Clicker
An automated Cookie Clicker tool.
Simple game built with devkitPro and libctru
A simple clone of cookie clicker
In Cookie Clicker, your goal is to generate as many cookies as possible. You can click on a big cookie icon to bake a cookie, and you can also purchase items that automatically generate cookies for you over time. We model the ability to click on the big cookie icon as an initial cookie generation rate. You can use the cookies you have generated as currency to purchase various items that increase your cookie generation rate. Items can be purchased multiple times, but after each item purchase, the item’s cost will increase at an exponential rate, given by , where is the cost of the first item and is the cost of item . In the actual game, . The real game has no explicit end condition, but in this paper we define two possible end conditions: reaching a certain number of cookies, or reaching a certain cookie generation rate .
Cookie Clicker falls into a broader class of popular online games called incremental games or idle games [Wik14], in which the primary mechanic of the game is acquiring income and spending that income on income generators in order to acquire even more income. Some other well-known games in this genre include Adventure Capitalist, Cow Clicker, Clicker Heros, Shark Souls, Kittens Game, Egg Inc., and Sandcastle Builder (based on the xkcd comic 1190, Time).
In most of this paper, we will assume that you start with cookies and that the initial cookie generation rate from clicking on the big cookie icon is . We will describe each item by a tuple , where denotes how much the item will increase your cookie generation rate, denotes the initial cost of the item, and denotes the multiplicative increase in item cost after each purchase. The case where for every item is a special case called the fixed-cost case. The goal of the game is to find the optimal sequence and timing of item purchases that optimizes some objective.
There are multiple possible objectives that we could want to optimize for, but we focus on the following two:
1. Reaching cookies in as little time as possible
2. Reaching a generation rate of in as little time as possible
We will usually begin by discussing the version of the problem and then explain how to extend our results to the version. A third natural objective is to maximize the number of cookies or the generation rate achieved given a total amount of time , and it can be solved by any algorithm that solves the first two variants using binary search on the values of and .
Our analysis of various versions of Cookie Clicker gives rise to interesting and varied results; refer to Table 1. First, we present some general results, such as the fact that the optimal strategy involves a Buying Phase where items are purchased in some sequence as quickly as possible, and then a Waiting Phase where no items are purchased.
We begin our version-by-version analysis by examining the case where exactly 1 item is available for purchase, and we present formulas describing how many copies of the item should be purchased in both the fixed-cost case and increasing-cost case.
Next, we analyze cases involving 2 items. In the 2-item fixed-cost case, we prove that the optimal solution always involves consecutively buying some number of copies of one item, followed by consecutively buying some number of copies of the other item.
Then, we analyze the case involving items. In the -item fixed-cost case, a weakly polynomial time dynamic programming solution can be used to find the optimal sequence of items to buy, and in the increasing-cost case, a strongly polynomial time dynamic programming solution can be used. Additionally, a greedy algorithm can be devised with an approximation ratio that approaches for sufficiently large values of .
Afterwards, we present negative results, including proofs of weak NP-hardness of the decision version of the problem of reaching a generation rate of R as quickly as possible, as well as for a version of Cookie Clicker that allows you to start with a nonzero number of cookies. Finally, we define a discretized version of Cookie Clicker where decisions regarding whether or not to buy an item happen in discrete time steps and prove strong NP-hardness for that version.
Python implementations of the dynamic programming solution and the greedy solutions to the General Cookie Clicker problem, and the dynamic programming solution to the Fixed-Cost Cookie Clicker problem, are available.222https://github.com/kaixiao/Cookie-Clicker
Result for version
Result for version
|1-Item Fixed-Cost with item [§2.1]||OPT takes time
to compute OPT
|OPT takes time
to compute OPT
|1-Item Increasing-Cost with item [§2.1]||OPT will stop Buying Phase after items
to compute OPT
|OPT will stop Buying Phase after items
to compute OPT
|2-Item Fixed-Cost with items where [§2.2]||OPT is of the form for large enough
to compute OPT, where
|OPT is of the form for a small number of ’s at the end for large enough .|
|-Item Fixed-Cost with items [§2.3]||to compute OPT using Dynamic Programming||to compute OPT using Dynamic Programming|
|-Item Increasing-Cost with items [§2.4]||
to compute OPT using Dynamic Programming
Greedy Algorithm has Approximation Ratio of
to compute OPT using Dynamic Programming
Weakly NP-hard by reduction from Partition
|-Item Increasing-Cost with items with Initial Cookies [§3.3]||Weakly NP-hard by reduction from Partition||Weakly NP-hard by reduction from version|
|-Item Increasing-Cost with items and Discrete Timesteps [§3.4]||Strongly NP-hard by reduction from 3-Partition||Strongly NP-hard by reduction from version|
Before proceeding to our main results, we develop some useful tools for finding optimal solutions for playing Cookie Clicker. We present these tools and show how they are applied to the 1-Item Case, but these tools are applicable to all versions of the game.
First, we can define a game state as a tuple where is the number of cookies you have and is the quantity of item that you have. In the 1-Item case, the tuple is just . Note that your current state in the game is entirely described by this tuple.
In general, the following claim is true.
If the next step of the optimal strategy involves buying an item at some point in the future, you should buy the item as soon as you can afford it.
Suppose that from a given game state, a strategy involves buying the th item seconds after you can afford it. Let denote the cookie generation rate at the game state. The net change in game state after these seconds is a gain of copy of item and a change in the amount of cookies by .
Then, consider the strategy that buys the th item as soon as you can afford it and waits seconds afterwards. In this case, the net change in game state after these seconds is a gain of copy of item and a change in the amount of cookies by . Thus, this new strategy results in the exact same result as the original strategy, except that it gains an extra cookies, which is strictly better. Thus, an optimal strategy that intends to buy an item as its next step must buy it as soon as it can be afforded. ∎
This claim tells us that the optimal strategy will always wait until it can purchase an item and purchase it immediately, or it will wait until the target number of cookies is reached. Thus, the problem boils down to jumping between game states in which you have cookies and need to decide between waiting to reach cookies or purchasing an item. This means that the only thing we need to keep track of to determine our game state is , the number of each item we have purchased. For example, in the 1-Item case, we only need to keep track of a single number . Thus, we can define to be the minimum amount of time needed to reach cookies from the game state .
From Claim 1.1, we can deduce that the optimal solution will have 2 phases. We will call them the Buying Phase, where the solution tries to buy items, and the Waiting Phase, where the solutions has bought all the items it needs and just waits until the items generate cookies. Every optimal solution can be represented by the sequence of items that should be bought in the Buying Phase.
Next, we define some general notation that will be useful in the future.
is the amount of time needed to buy the items , , , in order from an initial state with rate and rate gains, costs, and cost increases described by the vectors
and rate gains, costs, and cost increases described by the vectors, , and respectively.
In cases where , we may leave out the parameters in the notation, as they are fixed throughout the problem. Sometimes, we also leave out if it is clear what is being referred to.
It is often helpful to bound the value of . The following two results are true in general.
The following inequalities hold:
Let be the generation rate just before purchasing item . Note that for all , . Additionally, we know that . Then we have that
Next, we determine conditions for when buying an item is optimal. In general, we can determine an upper bound on the generation rate beyond which it will not be worth it to purchase any more items.
If your current cookie generation rate is and the items have rate increases and costs , you should stop buying items if and only if
To prove the result, we begin by proving an alternate form of the result for the 1-Item example.
If your current cookie generation rate is , you should buy an item with cost and rate increase if and only if
Suppose we are at a state where we have purchased items. Then, the optimal decision is either to purchase another item or to enter the Waiting Phase.
In the first case, the time taken is equal to
because a valid (but possibly not optimal) strategy from the state is to wait.
In the second case, the time taken is equal to .
Thus, if it is the case that
then we should go with the first strategy and purchase an item. Rearranging gives . This means that if is satisfied, purchasing the item is better.
Now we show that if is not satisfied, then waiting is better. Suppose that is not satisfied, so . Written in the form of , this inequality becomes . Note that for any rate and , the inequality still holds. Now, suppose that the optimal strategy from this point forward is to purchase items for some and then wait. Let and denote the price and cookie generation rate after item purchases from this point forward, and note that . Then the time taken to achieve this equals . Note that for any
Thus, we have that
Thus, if is not satisfied, then the optimal strategy is to wait. This completes the proof of the lemma. ∎
Isolating from Lemma 1.5 and reversing the statement, the following corollary holds.
If your current cookie generation rate is , you should stop buying items with cost and rate increase if and only if
Armed with the tools we developed in the previous section, we solve the 1-Item Cookie Clicker problem. Based on the results of the previous section, the optimal strategy is to purchase items for some as soon as each item becomes affordable and then wait until we reach cookies. The total time that this takes is
By Lemma 1.5, we know that if our current cookie generation rate is and the current cost of the item is , then we should stop buying the item when . After buying items, we have that and .
In the special case of , which we call the fixed-cost case, the inequality becomes
so , the number of items we should buy before stopping, is the smallest integer larger than . In this case, the total time the optimal solution takes is equal to
If , the inequality then becomes
In most reasonable cases, the term on the left hand side of the inequality is fairly small, so .
Now, we derive similar results for the rate-goal version of the problem, where the goal is to reach a final rate of cookies. Note that in this version, there is no Waiting Phase, so only the Buying Phase needs to be analyzed. The optimal strategy for the rate-goal version is quite simple: buy the item whenever possible until the goal rate is reached. The goal rate of is reached after purchases of the item.
Then, for the fixed-cost case where , the total time needed to reach the rate goal will be
And for the increasing-cost case, the total time needed to reach the rate goal is
In this section and the next, we analyze the case where all the ’s are equal to 1, which we call Fixed-Cost Cookie Clicker. This is a natural starting point, as it corresponds to the economic situation in which items are fixed in price due to enough supply existing. You can think of this game as modeling the problem of optimizing discrete investments.
In the 2 Item Cookie Clicker problem, our goal is to reach cookies as quickly as we can, and the 2 items available are described by the tuples and . These are defined analogously to the 1 item case. Without loss of generality, we can assume that . In this problem, we will also make the assumption that . This is because if the reverse inequality held, then buying copies of item 1 gives a higher rate increase than buying a single instance of item 2, which means that it will never be optimal to buy item 2 if is large enough333 must be large enough so that the effect of not being an integer is irrelevant in the long run.
We can apply the general claims from the 1 item analysis here, so we know from Claim 1.1 that the optimal strategy will jump between states where we have cookies, and that there is a Buying Phase and a Waiting Phase. As before, we can represent every optimal solution by the sequence of items that should be bought in the Buying Phase.
We now solve this problem. We will show that the sequence of items in the Buying Phase must be when is large enough. Then, finding the optimal solution simply involves figuring out when to stop buying item 1 and when to start buying item 2, which can be determined in polynomial time.
To help us solve our problem, we will define the following.
The efficiency score of an item of cost and rate increase when you have generation rate is .
If you plan to buy both items consecutively, you should always buy the item with the lower efficiency score. In particular, let . Then, if , you should purchase item followed by item , and if , you should purchase item followed by item .
The efficiency score of an item dictates whether buying item then item is better than buying item then item .
Suppose we have generation rate . Then the cost of buying item then item is equal to and the cost of buying item then item is equal to . If , we can rearrange the inequality to get that
On the other hand, if , then the reverse is true. ∎
Now, suppose that we have some optimal solution represented as a sequence of 1’s and 2’s. Now, we know that until the rate reaches , we will never have a 2 followed by a 1. Similarly, after the rate passes , we will never have a 1 followed by a 2. Thus, the final sequence must be of the following form.
Somewhere in the middle of the sequence of 2’s, the generation rate reaches .
Now, we will show that for large enough , there will be no sequence of 1’s at the end of the optimal solution.
Let If , then the optimal solution will have no 1’s at the end.
Suppose for the sake of contradiction that there are 1’s at the end of the sequence representing the optimal solution for some . We will show that replacing the final 1 with a 2 results in a better solution, which disproves the optimality of the original solution.
Denote that the rate before purchasing the final 1 in the optimal solution as .
The time it takes to buy the final 1 and then wait until the goal is reached is equal to . The time it takes to buy a 2 instead of the final 1 and then wait until the goal is equal to . We want to prove that
or equivalently that
Now, we know from Lemma 1.5 and the fact that the optimal solution bought the final 1 that
Similarly, because the optimal solution can not buy another 2 after the final 1, we know that
Combining the above two equations, we end up with
Thus, to prove (3), we just have to prove that
Now note that because the optimal solution can not buy another 1 after the final 1,
Because and , we can deduce that
Thus, . Using this, all we have to prove now is that
or equivalently that
But this is true because
Thus, we have shown that for large enough , the optimal solution will be of the form
where the 1’s only appear if the total generation rate at that point is less than the threshold . We can experimentally verify that the point at which the optimal solution transitions from 1’s to 2’s is not exactly , but is usually close to . An example of this is displayed in Figure 2. For every integer , we can consider the optimal strategy that starts off by buying exactly copies of item 1 then transitioning to item 2. Figure 2 plots the amount of time each optimal solution takes.
We then have the following corollary
Fixed-Cost Cookie Clicker for 2 Items can be solved in time, where and .
To solve Fixed-Cost Cookie Clicker for 2 Items, we just have to find the optimal number of 1’s to buy before transitioning to 2’s and subsequently solving the 1-Item Cookie Clicker game. Finding this optimal number involves maximizing a function of two bounded discrete variables (the number of 1’s to buy and the number of 2’s to buy), which can be done in polynomial time.
In particular, let us define
Then the function is unimodal in , because for a fixed , we are essentially solving 1-Item Cookie Clicker using the item 2. We can obtain rough upper bounds for each of and , which we denote by and , by considering an instance of 1-Item Cookie Clicker with just item 1 or just item 2. Then, we can find an optimal solution in [DL05]. ∎
Next, we analyze the rate-goal version of Fixed-Cost Cookie Clicker for 2 Items and obtain a similar result about the structure of solutions.
For large enough , any optimal solution to the rate-goal version of Fixed-Cost Cookie Clicker for 2 Items must be of the form
where the number of ’s at the end is upper bounded by .
This theorem lets us restrict the space of possible solutions, and we can use brute force on the number of ’s at the end to obtain an algorithm that finds the optimal solution in the same amount of time as in the version, up to a multiplicative factor corresponding to the brute force search.
First, we know using the same swapping argument as before that the solution must be of the form
Thus, we only have to show that the number of ’s at the end must be small. The primary idea is that if item 2 is indeed more efficient, you should be able to replace copies of item 1 with a single copy of item 2, giving a greater rate increase in less time, which can only improve your solution. This argument isn’t always easy to prove, depending on the costs and rate increases of items 1 and 2. Thus, we will instead argue that buying copies of item at the end is worse than buying copies of item of item 2 for some positive integers and . In particular, the minimal such that the rational number satisfies the equation
is sufficient for large enough . It is difficult to find the minimal , but we know that if we choose an such that , then there must exist some that satisfies the inequality and is less than . Thus, one valid choice of is , and is then upper bounded by . This upper bounds the number of trailing 1’s we can have in any solution.
To prove the above claim, consider replacing trailing copies of item 1 in some solution with copies of item 2 to produce the new solution :
The rate increase from the trailing item 1’s in is , and the rate increase from the trailing item 2’s in is . Based on our definition of and , we know that the rate increase from the trailing item 2’s in is higher, so will also reach the goal rate of .
Now, we just have to show that the solution takes less time than the solution . Suppose that the generation rate before purchasing the copies of item 1 is . The amount of time it takes to buy the copies of item 1 in is
The amount of time it takes to buy the copies of item 2 in is
where both inequalities are derived from Lemma 1.3.
Thus, we just have to show that
or equivalently, that
Let be the fraction on the right hand side of (4). The denominator of is positive from the definition of and , so as long as is large enough, buying copies of item 2 at the end also takes less time than buying copies of item 1. Thus, as long as , the optimal solution will have at most trailing 1’s. ∎
A natural follow-up is to extend this problem from 2 items to items. Here, we present a weakly polynomial time dynamic programming solution and discuss an attempt using local optimizations to achieve a faster runtime.
In the fixed-cost case, the items do not change in price over time. Thus, the only thing that determines the game state is the current generation rate. Using Lemma 1.4, the final generation rate is bounded above by . Let DP denote the minimal amount of time needed to reach a goal of from a state where you have cookies and generation rate . The dynamic program can be solved by the following recurrence:
where the term corresponds to entering the Waiting Phase after achieving a generation rate of . If there are items, then solving the original problem, which is equivalent to finding the value of DP, takes .
For the rate-goal version of this problem, we can define the same subproblem DP, and the recurrence can be modified slightly to
Again, the final problem reduces to finding DP, and the total runtime is .
One idea for a faster algorithm is to use local optimizations on a given solution sequence to try to obtain a better one.
As we saw from the analysis of Fixed-Cost Cookie Clicker for 2 Items, one example of a local optimization is a “swap” that involves swapping consecutive elements in a solution if doing so improves the solution. Another natural local optimization, which we saw in the proof of Theorem 2.3, was the replacement of one item in a solution sequence with another.
Thus, to try to solve this problem, we tried using random local optimizations on a randomly generated initial solution until it reached a point where local optimizations could no longer improve the solution. The local optimizations we used included:
Adding an item to the solution at a specific index
Deleting an item from the solution at a specific index
Replacing one item with a different item at a specific index
Moving an item from one index to another. If an item is moved from index to , this is equivalent to a “swap”.
Sorting the solution so that the cheapest items come first
In general, these local optimizations would improve the initial solution sequences. In some cases these local optimizations would result in a final solution that matched the globally optimal solution computed using dynamic programming. However, in other cases, these local optimizations get stuck at a local optimum, which are points that are worse than the global optimum, but can not be improved any further using any of these local optimization operations. Thus, this leads us to the conclusion that our list of local optimizations is insufficient for finding a global solution, and any proof that relies solely on these operations will not be able to show global optimality of a solution.
From this point forward, we assume that the ’s, the cost increase rates, all satisfy . This is how the original Cookie Clicker game works, and is also a reasonable assumption. It corresponds to the economic situation in which items increase in price due to limited supply.
In this section, we present a dynamic programming solution which finds optimal solutions in time.
For simplicity, let us solve the problem for the case with items first. Recall that your state in the game is completely described by the tuple , where denotes the quantity of item you have purchased. Note that it will never be worth it to buy an item if the item costs more than the goal . This gives us upper bounds on and , namely, . This motivates us to define
Let the cookie generation rate at state be represented by . From the state , the optimal strategy is one of the following three choices: entering the Waiting Phase, buying item 1, or buying item 2. We can then derive the recurrence
corresponding to each of those three choices. If and are the upper bounds for and respectively, then we can revise the recursive formulas for and for any and in the proper range to only correspond to two choices (e.g. for , you can either buy item 2 or wait). We only need to initialize the value and then use the recurrence to fill out the rest of the dynamic programming table. Finally, our answer is .
Initializing the single boundary value takes time. Then, filling out the rest of the table takes time, so the total runtime is .
This dynamic programming approach can easily be extended to the -item problem. As before, one can note that the game state of the -item problem is described entirely by the -tuple , where is the quantity of item that you have purchased. We can use the same upper bound . Let denote the upper bounds for each .
We can similarly define to be the minimum time it takes to reach M from the state . Then, filling in any square in the grid involves checking the solutions the adjacent squares and doing an computation for each adjacent square. In total, this takes time. The only square we need to initialize is the corner . Thus, the total time complexity of this program is .
For the rate-goal version of the problem, we can also use dynamic programming to compute the optimal solution, but the complexity is significantly worse. We also define to be the minimum time it takes to reach from the state , but we can only obtain a naive upper bound for each of . This upper bound does not have a like the upper bound for the version of the problem because we can not take advantage of the fact that costs increase. No matter how costly the items get, we still have to get to the final rate . Using this upper bound, this dynamic programming approach takes .
One greedy solution that arises naturally in normal gameplay involves buying the item that has the highest rate increase to cost ratio . This is the calculation that most human players do when playing the game unaided. For the 2 item case, given most reasonable setting of the parameters, this approach actually performs fairly well. However, for certain settings of the parameters, this approach can be quite bad. For example, take . In this case, the second item has a much higher rate increase to cost ratio, which means that the natural greedy solution would save up for a long time to purchase the second item. However, it is much better to purchase the first item and increase your generation rate incrementally.
As we saw in the analysis of the fixed-cost case, the efficiency score was a helpful metric to determine which item to buy. Another idea for a greedy algorithm is to compute the efficiency score of each item and always choose the item with the lowest efficiency score. This approach is generally very close to optimal. In fact, we can prove approximation guarantees for this greedy solution.
Once again, we will begin with the 2 Item case. We derive an approximation ratio for the Efficiency Score Greedy Solution that approaches for sufficiently large . Specifically, we will prove the following theorem.
The Efficiency Score Greedy Solution obtains an approximation ratio of for sufficiently large .
The main idea is to use two propositions:
Proposition 1: Before you’re anywhere close to reaching the goal , you’ll want to purchase at least one more copy of item 1 and at least one more copy of item 2
Proposition 2: When is “big enough”, means that the most efficient item is locally and globally optimal
Using these two propositions, the greedy solution can be compared to the optimal solution in three phases.
Phase 1: In this phase, the greedy algorithm reaches a certain generation rate , which corresponds to the “big enough” referred to in Proposition 2. “Big enough” is quantified in Claim 2.7. The time taken is labeled as .
Phase 2: The optimal algorithm will buy some amount of each of the 2 items, reaching copies of item 1 and of item 2 (the game state ) near the tail end of the Buying Phase. From this point onwards, the optimal algorithm will only purchase one type of item — this is Phase 3.
We will show that the greedy algorithm also reaches the state . The interval between the end of Phase 1 and reaching the state is Phase 2. In this phase, the greedy algorithm and optimal algorithm would be equivalent from the same starting state444In the final analysis, we compare the time needed by the optimal algorithm starting from a worse state than the greedy one, so the greedy solution actually takes less time in Phase 2.. If we let denote the amount of time the greedy algorithm uses in Phase 2, and we let be the amount of time optimal algorithm takes to reach from , then .