We consider the problem of identifying the realization of a discrete random variableby repeatedly asking questions of the form: “Is x the identity of X?”. This problem has been extensively studied by cryptanalysts who try to identify a secret key by exhaustively trying out all possible keys, where it is usually assumed that the secret key is drawn uniformly at random. We consider an -tuple drawn from an i.i.d. source, on a finite alphabet where represents the corresponding categorical distribution, which is not necessarily uniform. We measure security against a brute-force attacker who knows the source statistics completely, and who would query all the secret strings one by one until he is successful.
Denoting the number of guesses by , the optimal strategy of the attacker that minimizes the expected number of queries is to guess the possible realizations of
in order of decreasing probability under. Massey  proved that the Shannon entropy of , , is a lower bound on the rate of growth of the expected guesswork, yet there is no upper bound on in terms of . Arıkan 
proved that when we consider a string of growing length whose characters are drawn i.i.d, the positive moments of guesswork associated with the optimal strategy grow exponentially, and the exponents are related to the Rényi entropies of the single letter distribution:111In this paper, denotes the natural logarithm.
where the Rényi entropy of order is
Note that recovers the Shannon entropy. We also use the notations and
interchangeably to refer to the Rényi entropy of a string drawn from a source with parameter vector. Although these connections have been extended to more general stochastic processes [3, 4], in this paper, we focus on i.i.d. processes for the sake of clarity of presentation.
Christiansen and Duffy  showed that the sequence satisfies a Large Deviations Principle (LDP) and characterized its rate function, . Beirami et al. [6, 7] showed that can be expressed as a parametric function of the value of a “tilt” in a family of tilted distributions.
We remark that when the metric of difficulty is the growth rate in the expected number of guesses as a function of string length, the challenge for the adversary remains the same even if the adversary does not know the source statistics [8, 9].
In this paper, we first show a counter intuitive result that the average guesswork increases when the source becomes “less uniform” if the user is subject to a total entropy budget on the secret string. Next, we introduce a natural notion of total guesswork budget on the attacker and show that the probability of success of an adversary subject to a total guesswork budget increases when the source becomes “less uniform,” which is consistent with our intuition of choosing uniform passwords. We will formalize these notions in the rest of this paper.
Ii Problem Setup
Given a finite alphabet , a memoryless (i.i.d) source on is defined by the set of probabilities for all , where and . Hence, is an element of the -dimensional probability simplex. We define as the open set of all probability vectors such that for all , which also excludes the uniform source
The tilt operation plays a central role in the analysis, and is the basis for many of our derivations:
Definition 1 (tilted of order ).
For any , define as the “tilted of order ”, where , where for all is given by
Definition 2 (tilted family of ).
Let denote the “tilted family of ” and be given by
Observe that is a continuum of stochastic vectors in the probability simplex. Thus, the tilted family of a memoryless string-source with parameter vector is comprised of a set of memoryless string-sources whose parameter vectors belong to the tilted family of the vector , i.e., .
Definition 3 (high-entropy/low-entropy members of tilted family of ).
Let and denote the sets of high-entropy and low-entropy members of the tilted family of , respectively, and be given by:
Figure 1 depicts the probability simplex of all possible ternary parameter vectors, . The yellow star represents the distribution . Note that the tilted family of is parametrized by . At , we get the uniform distribution and as , we get to the degenerate case of . The high-entropy and low-entropy members of the tilted family of are represented by blue and red, respectively. Note that all distributions in the high-entropy set, , have Shannon entropies higher than that of and are closer to the uniform distribution in the KL divergence sense . Hence, the higher entropy members of the tilted family are “more uniform” than the lower entropy members of the tilted family.
Definition 4 (entropy budget per source character).
Let denote the entropy budget per source character such that the user is required to choose a secret string from an i.i.d. process with parameter vector with .
The concept of a total entropy budget on the entire secret string is a natural one or the user would choose an arbitrarily complex secret string. We use the entropy budget per source character defined above to ensure that the user is subject to the same total entropy budget by adjusting the length of the secret string for a fair comparison between string sources that have different entropy rates.
Iii Positive Moments of Guesswork
We first consider choosing strings with the same total (Shannon) entropy budget and measure security in terms of the positive moments of guesswork. If two sources have different entropy rates, we adjust the comparison by drawing a longer string from the lower entropy source. Formally, let us consider two sources with parameter vectors and on alphabet . Further, let and be the entropy rates of the two sources. Let the entropy ratio be
Without loss of generality, throughout this paper we assume that , and hence . The user is given the option to choose a secret string from either of the two sources. For a fair comparison, we assume that the entropy of the two strings is the same, . That is
To compare the growth rates of the positive moments of guesswork, in light of (1), we compare and . This will in turn impose the same total entropy budget on the strings drawn from the sources with parameter vectors and .
For a parameter vector , let an information random variable be defined as one that it takes the value with probability for all . We need one more definition before we can state the result of this section:
Definition 5 (skewentropy condition (SEC)).
Note that varentropy has been studied extensively and naturally arises in the finite block length information theory [10, 11], and more recently in the study of polar codes . To the best of our knowledge, skewentropy has not been studied before, and we provide some properties of the SEC in Section V.
Equipped with this definition, we provide an ordering of the sources that belong to the same tilted family.
The proof is provided in the appendix. Theorem 11 provides a natural ordering of sources that belong to the same tilted family. The “less uniform” low per-character entropy members of the tilted family take exponentially more number of queries, on the average, to breach compared to their more uniform higher per character entropy counterparts.
Let denote the uniform source. Then for any , and any ,
Corollary 2 suggests that, of all sources whose parameter vectors are in the (interior of the) probability simplex, the uniform source is the easiest to breach in terms of the positive moments of guesswork when the user is subject to a total entropy budget. This is in contrast to our intuition that more uniformity provides better security.
Iv Probability of Success subject to a Guesswork Budget
In this section, we put forth a natural notion of total guesswork budget, leading to a security metric consistent with our intuition. Similar to the case of an entropy budget, we need to define guesswork budget per source character for our analysis.
Definition 6 (guesswork budget per source character).
Let denote the guesswork budget per source character, such that is the total number of queries that the inquisitor can make in order to identify a secret string of length .
Note that by this definition, the inquisitor is supposed to possess the resources for querying an exponentially growing number of strings (with the sequence length). In particular, corresponds to an adversary who is capable of querying all of the possible outcomes of the source to successfully identify the secret string with probability .
If , then
and if , then
Recall that Arıkan  showed that the growth rate of the moments of guesswork is governed by atypical sequences resulting in the appearance of the Rényi entropies in the expression. On the other hand, Lemma 1 states that the cutoff for the adversary to be successful with high probability is still governed by the Shannon entropy (as intuitively expected).
In the regime where we would like to study the behavior of correct guessing. The next lemma relates the exponent of an exponentially large number of possible guesses to the LDP rate function.
Hence, , and a larger directly implies a more secure source against a brute-force attacker who is subject to a guesswork budget for a fixed . We use the above rate function as the metric for comparing two string-sources given a total guesswork budget, naturally defined as .
Using the notion of the tilt, we can represent the rate function as a parametric function of for a family of tilted distributions. The rate function, , associated with can be directly computed as :
for . This characterization plays a central role in our derivations.
Recall that we adjust the string lengths in order to make sure that the secret string chosen by the user is subject to a given total entropy budget. As the idea of the total guesswork budget is that the adversary can make a fixed number of queries regardless of the source from which the user is choosing the password, we compare the sources in terms of the probability of success subject to an adjusted guesswork budget per source character (see (12)). To keep the total guessing budget of the adversary the same, i.e., we must adjust the guesswork budget per source character as follows:
In light of (14), we compare with for sources with parameter vectors and .
We are now ready to provide our results on the adversary’s probability of success.
Let . For any ,
if and only if satisfies the SEC (see Definition 5).
We remark that the same SEC appears to be the crucial quantity for the statement of Theorem 15 to hold. This theorem implies that when the adversary is subject to a guesswork budget (i.e., he can only submit queries to identify a secret string of length ) for some , then the chances of correctly identifying the random string produced by a “more uniform” high per-character entropy member of the tilted family is exponentially smaller than that of the less uniform low per-character entropy source belonging to the same tilted family so long as the source satisfies the SEC when the user is subject to the same total entropy budget and the adversary is subject to the same total guesswork budget. In particular, the uniform source is the most secure against such an adversary subject to a guesswork budget:
Let denote the uniform information source. Then, for any and , we have
We remark that these security guarantees are against an adversary that is not powerful enough to be able to explore the entire typical set rendering his chances of success exponentially small. The “more uniform” sources provide an exponentially smaller chance to such an adversary to be successful.
We emphasize that the implications of Theorems 11 and 15 are in stark contrast to each other. On the one hand, more uniformity results in an exponential decrease in the number of queries expected of an adversary to correctly identify a secret string when the user is subject to a total entropy budget (Theorem 11). On the other hand, more uniformity decreases the chances of an adversary in identifying the secret string when the adversary’s power is limited by a total guesswork budget as well (Theorem 15).
V Properties of the SEC
Noting that SEC introduced in Definition 5 is a new concept, we study this condition in more detail in this section. Let us start with the binary memoryless sources.
Let . Further, let . Then,
The next theorem is our main result for binary memoryless sources:
Any satisfies the SEC.
While Theorem 5 shows that all binary memoryless sources satisfy the SEC, the same argument does not extend to larger alphabets.
For any there exists , such that does not satisfy the SEC.
Despite the negative result in Theorem 6, we show that sources that are approximately uniform satisfy the SEC for any alphabet size. Here is the key result for such sources:
Suppose that is such that
Then satisfies the SEC.
As a corollary, we state the condition more explicitly in terms of ’s.
Suppose that is such that
Then, satisfies the SEC.
Figure 2 depicts the set of ternary distributions that do not satisfy the SEC. As can be seen, source close to uniform satisfy the SEC while sources that are close to uniform on a two-dimensional alphabet while almost missing the third character in the alphabet do not satisfy the SEC.
Vi Numerical Experiments
In this section, we provide some numerical experiments. We compare several binary sources, where is the source parameter vector. The parameter vectors used for the experiments are listed in Table I. The length and the parameter vector are chosen such that nats for all of the pairs. Although the theorems proved in this paper are of asymptotic nature, we have chosen to run experiments on finite-length sequences instead to emphasize the applicability of the results even in very short lengths. As can be seen in Fig. 3, as the entropy rate of the source decreases, the moments of guesswork increase exponentially subject to the same entropy budget. On the other hand, as shown in Fig. 4, as the entropy rate of the source decreases, the chances of an adversary subject to a fixed total guesswork budget increases, which is consistent with our intuition.
In this paper, we studied guesswork subject to a total entropy budget. We showed that the conclusions about security deduced from the analysis of the average guesswork could be counter-intuitive in that they suggest that the uniform source is not the strongest source against brute-force attacks. To remedy the problem, we introduced the concept of total guesswork budget, and showed that if the adversary is subject to a total guesswork budget, the uniform source provides the strongest security guarantees against the brute-force attacker, which is consistent with our intuition.
This is equivalent to showing that for all ,
for all . Let , and hence . The statement above is in turn equivalent to showing:
It is straightforward to show that (76) is equivalent to
Finally, we prove the following statement that is equivalent to (24):
This is equivalent to showing:
For all , we have
See  for the proof.
For all , we have
See  for the proof.
For all , we have
For any ,
where is defined in (10).
Let us recall that for some . We can find and in the domain of each rate function such that the derivatives of the rate function are both equal to a constant . It follows from  that:
where . We focus on , and hence . Note that , (equivalently ) corresponds to the coinciding zeros of both rate functions. Once again recalling that the rate functions are convex, proving is equivalent to showing that (as defined in (34)) for all . This is in turn equivalent to showing:
This is equivalent to:
It is straightforward to show that (36) is equivalent to
Finally, we prove the following statement that is equivalent to (37):
This is equivalent to showing:
For all , we have
For any , we have
For any , we have
For any , we have
Note that as both sides are equal and the limit of their derivatives are equal as well, while the second derivative of the left hand side is equal to completing the proof. ∎
For any , we have
The proof is similar to that of Lemma 11. ∎
Let be such that
where is the binary entropy function given by
The calculation of is straightforward by noting that this is a mixture of two uniform sources on alphabets of size and . To calculate , we have
Finally, to calculate , similarly to the calculations for , we get
establishing the claim. ∎
Let be drawn from . Further, let
Hence, by definition, and . Then, the condition in (20) would ensure that . Noting that the uniform distribution is excluded in , and hence the varentropy is nonzero, we apply Lemma 14 (with ) to obtain that
This is a sufficient condition for the SEC to hold, completing the proof. ∎
Let be a random variable supported on for some Further, let and . Then,
It is straightforward to show that is maximized if
for some , which in turn leads to ∎
-  J. L. Massey, “Guessing and entropy,” in Information Theory, 1994. Proceedings., 1994 IEEE International Symposium on. IEEE, 1994, p. 204.
-  E. Arıkan, “An inequality on guessing and its application to sequential decoding,” Information Theory, IEEE Transactions on, vol. 42, no. 1, pp. 99–105, 1996.
-  D. Malone and W. G. Sullivan, “Guesswork and entropy,” IEEE Trans. Inf. Theory, vol. 50, no. 3, pp. 525–526, Mar. 2004.
-  C. E. Pfister and W. G. Sullivan, “Renyi entropy, guesswork moments, and large deviations,” IEEE Trans. Inf. Theory, vol. 50, no. 11, pp. 2794–2800, Nov. 2004.
-  M. M. Christiansen and K. R. Duffy, “Guesswork, large deviations, and shannon entropy,” Information Theory, IEEE Transactions on, vol. 59, no. 2, pp. 796–802, 2013.
-  A. Beirami, R. Calderbank, M. Christiansen, K. Duffy, A. Makhdoumi, and M. Médard, “A geometric perspective on guesswork,” in 53rd Annual Allerton Conference (Allerton), Oct. 2015.
-  A. Beirami, R. Calderbank, M. Christiansen, K. Duffy, and M. Médard, “A characterization of guesswork on swiftly tilting curves,” preprint, 2017.
-  R. Sundaresan, “Guessing under source uncertainty,” IEEE Trans. Inf. Theory, vol. 53, no. 1, pp. 269–287, Jan. 2007.
-  A. Beirami, R. Calderbank, K. Duffy, and M. Médard, “Quantifying computational security subject to source constraints, guesswork and inscrutability,” in 2015 IEEE International Symposium on Information Theory Proceedings (ISIT), Jun. 2015.
-  V. Strassen, “Asymptotische abschätzungen in shannons informations theorie,” in Trans. Third Prague Conf. Inf. Theory, 1962, pp. 689–723.
-  Y. Polyanskiy, H. V. Poor, and S. Verdú, “Channel coding rate in the finite blocklength regime,” IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2307–2359, 2010.
-  E. Arıkan, “Varentropy decreases under the polar transform,” IEEE Transactions on Information Theory, vol. 62, no. 6, pp. 3390–3400, 2016.