Risk-Limiting Bayesian Polling Audits for Two Candidate Elections

by   Poorvi L. Vora, et al.

We propose a simple common framework for Risk-Limiting and Bayesian (polling) audits for two-candidate plurality elections. Using it, we derive an expression for the general Bayesian audit; in particular, we do not restrict the prior to a beta distribution. We observe that the decision rule for the Bayesian audit is a simple comparison test, which enables the use of pre-computation---without simulations---and greatly increases the computational efficiency of the audit. Our main contribution is a general form for an audit that is both Bayesian and risk-limiting: the Bayesian Risk-Limiting Audit, which enables the use of a Bayesian approach to explore more efficient Risk-Limiting Audits.



page 1

page 2

page 3

page 4


A Unified Evaluation of Two-Candidate Ballot-Polling Election Auditing Methods

Counting votes is complex and error-prone. Several statistical methods h...

The Athena Class of Risk-Limiting Ballot Polling Audits

The main risk-limiting ballot polling audit in use today, BRAVO, is desi...

Risk-Limiting Tallies

Many voter-verifiable, coercion-resistant schemes have been proposed, bu...

Lazy Risk-Limiting Ballot Comparison Audits

Risk-limiting audits or RLAs are rigorous statistical procedures meant t...

k-Cut: A Simple Approximately-Uniform Method for Sampling Ballots in Post-Election Audits

We present an approximate sampling framework and discuss how risk-limiti...

Voter Perceptions of Trust in Risk-Limiting Audits

Risk-limiting audits (RLAs) are expected to strengthen the public confid...

Next Steps for the Colorado Risk-Limiting Audit (CORLA) Program

Colorado conducted risk-limiting tabulation audits (RLAs) across the sta...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The framework of risk-limiting audits (RLAs), as described by Lindeman and Stark [1], formalizes a rigorous approach to election verification. The purpose of an audit is to require a full hand count if the outcome is wrong; the risk is the rate at which it fails to do so. A RLA is one that guarantees that the risk will be smaller than a pre-specified bound.

Computing the risk is tricky, however. If the audit rule and parameters are fixed, the risk depends on the (unknown) true tally of the election. Smaller margins in favor of the declared loser make it harder to detect that the outcome is wrong, and correspond to larger values of the risk. An audit typically uses a single measure to represent the varied values the risk can take as the underlying (unknown) tally varies.

  • A RLA uses as its measure the largest possible risk, corresponding to the wrong outcome which is hardest to detect: either a tie or a margin of one vote in favor of the(a) declared loser.

  • The Bayesian audit, as described by Rivest and Shen [6], is a newer approach to election audits. It considers each possible margin in favor of the declared loser(s). The Bayesian measure, termed the

    upset probability

    , is a weighted average of the corresponding risks.

Each type of audit samples ballots, stopping when the risk measure is smaller than a guaranteed upper bound.

Depending on what the true tally is, there could be considerable difference between the measure, which is computed from the sampled ballots, and the true risk, which is unknown. While all we can compute is the measure, it is the true risk we care about.

  • The true risk is never larger than the RLA measure (maximum risk). The RLA stops when the maximum risk is smaller than the bound; the true risk will also be smaller than the bound.

  • The true risk of a close election could be larger than the Bayesian measure, the upset probability, which is an average risk. The Bayesian audit stops when the upset probability is smaller than the bound, but the true risk might still be too large. There is no way for the observer to know whether this is the case, because the true tally is unknown.

In addition to proposing the Bayesian approach, Rivest and Shen also propose the use of Pólya urn simulations to compute whether an audit should end or not. While the simulations are not efficient enough for real time use, they provide the only way we know to carry out the audit of a complex election. Traditional RLAs are more efficient for plurality elections and can be reduced to a comparison test as described in the classical work of Wald [7]. The CLIP audit of Rivest [4] is another RLA which may also be reduced to such a comparison, though the values are computed using simulations111Philip Stark has mentioned work in progress: a CLIP-like audit which does not use simulations..

The Bayesian framework is exceptionally promising as a means of designing efficient audits (requiring a small sample size). As we have seen above, important open research areas include (a) the characterization of the risk limit (or maximum risk) of a Bayesian audit and (b) improving its computational efficiency.

The following are important open questions regarding characterization of Bayesian audits: Is there a well-defined relationship between RLAs and Bayesian audits? What is the relationship between the largest risk (RLA risk measure) and the upset probability (Bayesian audit risk measure)? What is the closest election for which the upset probability is not smaller than the risk? That is, what is the closest election for which the upset probability is a reasonable stand-in for the true risk? Can Bayesian audits be designed to be comparable to traditional RLAs in computational efficiency? Also of interest are questions regarding audits that are both Bayesian and risk-limiting: What form might Bayesian risk-limiting audits (RLAs)—where the upset probability is identical to the largest risk—take? Could Bayesian RLAs be designed to use a smaller sample than traditional RLAs?

While we do not attempt to answer all the questions posed above, we present early results that should provide a basis for exploring the answers. In this paper, while restricting ourselves to a two-candidate election and polling audits, we view both audits in a single framework. Among Bayesian audits, we study only those with equal prior winning probabilities for each candidate. We expect that our results will be applicable in a straightforward fashion to comparison audits for two-candidate plurality elections as well, though that is work in progress.

1.1 Our contributions

Our contributions are as follow:

  1. We show that the Bayesian audit can be reduced to a simple comparison test between the number of votes for the winner in the audit sample and two pre-computed values for this sample size:

    • a minimum number of votes for the winner, , above which the election outcome is declared correct, and

    • a maximum number of votes for the winner, , below which the audit proceeds to a hand count.

    The values of and can be pre-computed, making the Bayesian audit a feasible real-time audit, as it is a simple look-up process at the time of drawing samples. If Bayesian audits for more complex elections—such as those with multiple candidates (we are still working on this)—could also be reduced to comparison tests, computational efficiency could be greatly improved.

  2. We show that the traditional RLA as described in [1] is a Bayesian audit with a prior that assumes that, if the election were correct, the announced winner would earn a fraction of the votes; if not, she would win half the votes. Such an incorrect outcome is the hardest possible to distinguish from a correct one, which is what makes the audit a RLA.

  3. Motivated by our understanding of the traditional RLA

    , and restricting ourselves to odd

    , we define a class of Bayesian audits that are also RLAs. An audit in this class may have any prior on tally values for a correct outcome, but the only allowed tally for an incorrect outcome is that corresponding to the closest possible election lost by the winner (in this case, a margin of one). We prove that this is a RLA. For such a prior, we show that the upset probability of the Bayesian audit is also the largest risk.

  4. We report on verification of:

    1. our comparison test for the Bayesian audit (1)

    2. the risk limit of the audit in (3).

We draw considerably from the proof approaches in [7].

1.2 Organization

This paper is organized as follows. Section 2 describes the model and establishes most of the notation. Section 3 describes the Wald Sequential Test [7], RLAs [1] and Bayesian audits [6]. Our contributions are to be found in sections 4 and 5. Section 4 presents our results on a simple form of the Bayesian audit and a new audit that is both Bayesian and risk-limiting, the Bayesian RLA. Section 5 presents look-up table forms of the audits and verification of the simple form of the Bayesian audit and the risk of a Bayesian RLA. Section 6 concludes. Acknowledgements are in section 7 and proofs in the Appendix.

2 The Model

We consider a plurality election with two candidates, voters and no invalid votes. We assume that is odd so the winner is well-defined. It is not hard to extend the results to even , we use odd for ease of exposition. We denote by

the random variable representing the true winner, by

an instance of , by and the announced winner and loser respectively and by the (true, unknown) number of votes obtained by . Thus if and only if .

A polling audit will estimate whether

is the true winner. Consider a sample of votes drawn uniformly at random: , , …, , , . The sample forms the signal or the observation; the corresponding random variable is denoted , the specific value . Let denote the number of votes for in the sample; then votes are for .

The audit computes a binary-valued estimate of the true winner from :

We will refer to the function as the estimator and as the estimate. The audit uses an error measure to compute the quality of the estimate.

  • If and the error measure is acceptable we are done (stop) and declare that the election outcome was correctly announced.

  • If and the error measure is acceptable we stop drawing votes and proceed to perform a complete hand count.

  • If the error measure is not acceptable we draw more votes to improve the estimate.

In computing , we can make two types of errors:

  1. Miss: A miss occurs when the announced outcome is incorrect, , but the audit misses this, and . We denote by the probability of a miss—given that the announced outcome is incorrect, the probability that the audit will miss this:

    is the risk

    in risk limiting audits. If the audit is viewed as a statistical test, with the null hypothesis being


    is the Type I error.

  2. Unnecessary Hand Count: Similarly, if , but , acceptance of the estimate would lead to an unnecessary hand count. We denote the probability of an unnecessary hand count by :

    If the audit is viewed as a statistical test, with the null hypothesis being ,

    is the Type II error.

3 Defining the audit

In this section, we describe three types of audits. We do not attempt to introduce any new ideas, but try to faithfully represent the existing literature.

  1. Wald Sequential Tests

    A classical approach is the Wald Sequential Test which limits both and .

    Definition 1: The Wald Sequential Test is the likelihood ratio test [7]:


    where is the likelihood ratio:

    and .

    Proposition 1 [7]: The Wald Sequential Test has and , and is a most efficient test achieving these bounds.

    (A most efficient test is one requiring the smallest sample size). An argument supporting Proposition 1 may be found in [7].

    Suppose the draws are independent (with replacement) and of the votes in the sample are for . To compute the expressions and required by the test, we need , the election’s true vote counts for , when and respectively. Assume that if wins, she wins with (greater than half) votes, and if she does not ( wins or the election is a tie), obtains (no more than half) votes, where and are integers, and and . With these assumptions, the Wald Sequential Test is easily seen to be [7]:


    We will refer to the test defined by (2) as the Wald Sequential Test.

    Corollary 1: When the only possible values of the true vote count, , are ( wins) or ( loses), the Wald Sequential Test has and , and is a most efficient test achieving these bounds.

    Proof: This follows from Proposition 1.

  2. Risk-Limiting Audits (RLAs) [1]

    The Wald Sequential Test requires prior knowledge of and , the values of when is the winner and loser respectively. Unless one performs a full hand count, however, one does not know the true value of . If and are misestimated, the true upper bounds on the risk and the probability of an unnecessary recount might not be and respectively.

    This is particularly important because the audit outcome is final (while the outcome is followed by a confirmatory full hand count). At the very least then, we should guarantee an upper bound on worst case errors when the audit outcome is . That is, we would like to bound the risk, independent of the true value of .

    A risk-limiting audit (RLA) with risk limit —as described by, for example, Lindeman and Stark [1]—is one for which the risk is smaller than for all possible (unknown) true tallies in the election (or—equivalently for the two-candidate election—all possible values of ). For convenience when we compare audits, we refer to this audit as an -RLA.

    There are many functions that would satisfy the -RLA criterion, and not all would be desirable. For example, the constant estimate always requires a hand count and is risk-limiting with , , . However, , and the average number of votes examined by the audit is ; this is undesirable.

    A more efficient example of an -RLA is the traditional RLA [1] based on Wald sequential tests:


    where depends on the fraction of votes declared for and is the desired upper bound on . We denote this the -traditional RLA and note that it is identical to the Wald Sequential Test. When , this is the BRAVO audit [2], which may be denoted the -BRAVO audit.

    Other RLAs include the CLIP audit [4] which may be expressed as a simple comparison test between the number of votes for the winner and a pre-computed value that depends on sample size.

  3. Bayesian Audits [6]

    Bayesian audits, defined by Rivest and Shen [6], assume knowledge of a priorprobability distribution on ; we denote this distribution by . In this model, the variable inherits a prior distribution from , because


    Further, given the sample , inherits a posterior distribution, , also known as the a posteriori probability of . The Bayesian audit estimate is the candidate that maximizes this probability (that is, the candidate for whom this value is larger), with the constraint that the probability of estimation error (the upset probability) is smaller than , a pre-determined quantity, .

    The (computational) Bayesian Audit assumes the audit draws votes without replacement and uses knowledge of to simulate the distribution on the unexamined votes, conditional on , using Pólya urns. The estimate is the estimate with the largest number of wins in the simulations, provided the fraction of wins is greater than .

    The drawing of votes without replacement becomes particularly important in a tight election, where low margins tend to require large samples; an audit with replacement could result in more than draws, while an audit without replacement would definitely stop in draws. Sampling without replacement makes far more efficient use of the information in a sample, and has since been adopted in the design of traditional RLAs as well, see, for example, [3].

    The three contributions of Rivest and Shen (sampling without replacement, the Bayesian approach for election audits and computational Bayesian audits) do not have to be used together: sampling without replacement can be used for any audit, and Bayesian audits do not have to be computed using Pólya urn simulations.

    We study the general Bayesian audit and do not restrict ourselves to Pólya urn simulations; this is particularly easy in the two-candidate election. We will refer to the general Bayesian audit as described above as the -Bayesian audit. Further, to avoid confusion, the term will always refer to audits without replacement. We do not explore Bayesian audits with replacement. Additionally, as mentioned earlier, we assume that

  4. Audits with and without replacement

    So that we may understand better all the differences between Bayesian audits and other audits, we provide expressions for both the Wald Sequential Test and the traditional RLA without replacement. We note that the likelihood ratio is the ratio of the probabilities of drawing a single permutation (denoted by the specific audit sequence of draws that represents) when the number of votes for is and respectively.

    That is,


    is the hypergeometric distribution: the probability of obtaining

    items with the desired characteristic when items are drawn from a total of items of which have the desired characteristic. In our case, the items are votes in the election, and those with the desired characteristic are votes for . Thus is the probability of drawing votes for in a sample of size drawn from the votes cast in the election, of which is the true number of votes for . Dividing this value by gives us the probability of drawing a particular sequence of votes of which are for . (The term will be common to the numerator and denominator in the likelihood ration and will cancel out).

    The Wald Sequential Test without replacement is:


    The reader may compare the above to (1) and (2).

    Similarly, the -traditional RLA without replacement is:


    where is the fraction of votes declared for . The reader may compare the above to (3). Note that, for , this is the -BRAVO audit without replacement.

4 Relationships Among the Audits

In this section we demonstrate relationships among the different types of audits described in the previous section. Some of the material presented is obvious, some might have been shown elsewhere. But we are not aware of this material appearing together elsewhere in similar form, and we believe Theorems 1 and 2, at the very least, are original.

4.1 A general expression for the Bayesian audit without replacement

In this section we derive a general expression for the Bayesian audit.

Theorem 1: The -Bayesian audit is of the form:


where is the ratio:



Proof: See the appendix for the proof. (Note that, for the summations in (7), it is that varies, while, for the hypergeometric distribution, it is

that varies. A normalizing factor in both the numerator and denominator—from the application of Bayes’ theorem—accounts for this, and cancels out.)

Readers may compare the expression of Theorem 1 with expression (5).

4.2 The Wald Sequential Test and the Bayesian Audit

Some Wald Sequential Tests are instances of a Bayesian audit. In this section, we make this relationship precise. Here, denotes the (discrete) Kronecker delta function which takes on the value for and is zero otherwise.

Corollary 2: The Wald Sequential Test without replacement is the -Bayesian audit for

Proof: The result follows trivially from (4), (6) and (7).

4.3 The traditional RLA as a Bayesian Audit

When the only possible values for are or , the value is the risk of the Wald Sequential Test. The risk of the test could be larger if the true value of is not one of these two. On the other hand, whatever the true value of , the risk of the Wald Sequential Test will not be larger than ; that is, the Wald Sequential Test is an -RLA. While we have not seen this result proven in the literature, it is well-known, and related to Theorem 2, which we prove later.

The traditional RLA (3) is a special case of the Wald Sequential Test, with and chosen according to various considerations. From Corollary 2, when the traditional RLA is also a Bayesian audit with .

Corollary 3: The -traditional RLA without replacement is the -Bayesian audit for

Proof: The result follows trivially from Corollary 2 and the fact that the -traditional RLA is the -Wald Sequential Test.

Note that the -BRAVO audit may not be represented as a special case of the above because the Bayesian audit as defined by Rivest and Shen requires . However, a more general definition of the Bayesian audit, where the probability of erring when the outcome is correct is zero and not equal to the probability of erring when the outcome is wrong, would correspond to the BRAVO audit for as above.

4.4 A General Rla

In this section, we see that we can define a general form of the RLA using the Bayesian model. To do so, we first examine in more detail the risk of an audit.

4.4.1 The Risk of any Audit

We first establish additional notation in order to represent the risk. Given any audit, consider the audit sample sequences for which the audit stops with . Denote this set of sample sequences by . Note that is not fixed because the number of samples drawn is not fixed, thus contains sequences of different lengths. Similarly, denote by the set of sample sequences for which the audit ends with .

Lemma 1: The -Bayesian audit has

Proof: See the Appendix for the proof.

Each audit we have covered (whether Bayesian, traditional risk-limiting or Wald Sequential) assumes a prior (the values of and for the Wald Sequential Test, for the traditional RLA and for the Bayesian audit). The choice of prior results in the sets and of sample sequences for which the audit stops—with or , respectively. A different prior—for the same audit—would result in different sets and , which are independent of the particular election itself, or the true tally. These sets define the audit. The true risk of an audit is a function of the set and the true tally when wins (the unknown value of ). In order to avoid confusion, we will not denote the true risk by , which has so far referred to an ex ante definition of the risk (one that does not take the true tally into consideration, but is derived using the assumed prior).

The true risk of the audit is the probability of drawing any of the sample sequences in , when the number of votes for is for some unknown . Denoting true risk, or ex poste risk, by , we have:


See Appendix for details.

4.4.2 The Risk-Limiting Bayesian Audit

We now describe a way to choose a set of points such that . That is, we describe an approach to obtaining an -RLA. For this purpose we first define the particular type of probability distribution on .

Given a prior of the vote count for election , define the risk-maximizing distribution corresponding to (denoted ) as follows.


Note that is a valid distribution for the vote count of an election.

Theorem 2: The -Bayesian Audit is an -RLA with for election with prior .

Proof: See Appendix.

Corollary 4: The -Wald Sequential Test, which is also the -traditional RLA, is an -RLA.

Proof: Follows from Corollary 2 and Theorem 2.

A more general version of Corollary 4, for the -traditional RLA is generally known to be true, and can be proven as above but we are not aware of a proof in the literature on election audits.

5 Computing Rlas and Bayesian Audits

We defined a general Bayesian RLA in the previous section. The prior is not a natural fit to computing using Pólya urns, however. In this section we describe how the Bayesian RLA may be pre-computed. Pre-computation improves the computational efficiency of a Bayesian audit, no longer constraining us to the use of Pólya urn simulations. As a consequence, we are also not restricted to beta distributions for the prior. We begin with the pre-computation of the traditional RLA, which follows from a classical result by Wald [7].

5.1 Audits and pre-computed look-up tables

We observe that the Wald, traditional RLAs and Bayesian audits may be defined in the form:


where and are determined by the specific audit.

For example, for the traditional RLA (3), is monotone increasing with because , and hence


Let be the smallest integer such that

and the largest integer such that:

and hence:




For the traditional RLA without replacement, simple algebra demonstrates that is monotone increasing with and is the smallest integer such that

and the largest integer such that:

For the Bayesian audit (6) too, one may show, as one may expect, that is monotone increasing with . is the smallest integer such that

and the largest integer such that:

5.2 Experimental Verification

We performed experiments to: (a) generate lookup tables using the expressions we derived for the general Bayesian audit, as described in Theorem 1 and section 5.1; (b) verify the lookup tables; (c) generate lookup tables for the Bayesian RLA we propose and verify that the maximum risk is as expected and (d) compare the number of samples required to stop the various audits.

5.2.1 Generation of Lookup Tables for the Bayesian Audit

We used the derived expressions for the Bayesian audit to form a look-up table for values of given values of (see Table 1), assuming the beta distribution prior with pseudo-counts of for each candidate ( is proportional to ), an election with votes, and an escalating audit with sample sizes escalating by a factor of 2—beginning at 200 and ending at 51,200. We generated values of for each of the nine values of and , see Table 1.

We used the computed values of to simulate 1,000 audits of a tied election with votes, and computed the fractional number of times the audit stopped (declaring the outcome correct); this is an estimate of the maximum risk of the audit over all possible values of the tally. We observe that the maximum risk is many times the upset probability. In unpublished simulations performed independently by us, Ottoboni and Rivest, the maximum risk is even larger for finer audit samples.

Upset Max. Audit sample size,
Prob. Risk 200 400 800 1,600 3,200 6,400 12,800 25,600 51,200
0.1 0.416 110 213 419 826 1,636 3,250 6,468 12,889 25,702
0.05 0.236 112 217 424 833 1,646 3,264 6,487 12,914 25,731
0.02 0.138 115 221 429 841 1,658 3,280 6,509 12,942 25,763
0.01 0.061 117 224 433 847 1,665 3,291 6,523 12,961 25,785
0.005 0.028 119 226 437 852 1,672 3,300 6,537 12,978 25,804
0.002 0.011 121 229 441 858 1,681 3,312 6,553 12,999 25,828
0.001 0.007 122 231 444 862 1,686 3,320 6,564 13,014 25,845
Table 1: Values of computed for the Bayesian audit with and proportional to

5.2.2 Verification of the Bayesian Audit Expression

To verify our results, we compared the maximum risk estimates obtain using the lookup tables to those computed using Rivest’s public software library [5] with 10,000 inner trials for the Bayesian simulations (10,000 simulations given an audit sample, to estimate the Bayesian posterior and the upset probability) and 10,000 outer trials (10,000 instances of the audit) on a tied election with . We had access to the following data222Thanks to Ronald L. Rivest for providing the results of these simulations that he had carried out for a different purpose.:

  1. The final values of —the number of votes for in the sample—and —the size of the sample—for each of 10,000 instances of the Bayesian audit, for and .

  2. The fractional number of times the audit stopped (maximum risk) over 10,000 instances, for .

We know that the final values of ((1) above) are values for which the Bayesian audit computed with Pólya urn simulations stopped. We compare these values of with the corresponding values of computed by us to determine if the audit using the look-up table would also have stopped. We found that the error rate between the two audits was 0.0855 for and 0.0118 for , which indicates considerable general agreement.

In both cases most of the errors occur at the sample size of , when the Bayesian audit goes to a full hand count, while our expression predicts that it should stop. This is because the Bayesian audit, based on probabilistic simulations, may take an audit with votes for the winner to a full hand count, and also stop the audit for a sample with votes for the winner, when

. We expect that this variance is a function of the number of inner trials. On the other hand, the behaviour of the look-up table audit is a deterministic and monotonic function of the value of

, and if a sample with votes goes to a full hand count, so would any sample with votes, for all .

We compared (2) above to our estimates of the risk reported in Table 1. Our results were similar (see Table 2), and differences are likely attributable to the difference in the number of simulations (10,000 and 1,000) and in the finiteness of the number of simulations; that is, the values in both cases are simply estimates.

Upset Probability Risk Limit of Bayesian Audit Risk Limit Estimated using Lookup Table
0.1 0.4168 0.416
0.05 0.2531 0.236
0.02 0.1244 0.138
0.01 0.0634 0.061
0.005 0.0372 0.028
0.002 0.0154 0.011
0.001 0.0083 0.007
Table 2: Risk limit estimates for the seven-tier Bayesian audit for and proportional to

5.2.3 Example Bayesian Risk Limiting Audits

We computed two Bayesian RLAs.

  1. , risk measures of 0.1 and 0.005

    We computed Bayesian RLAs for , risk measures of 0.1, 0.05 and 0.005, and an escalating audit with sample sizes escalating by a factor of 2—beginning at 200 and ending at 51,200, and a prior that is uniform on tallies favoring the winner and concentrated on a margin of one for tallies favoring the loser. In Table 3, we compare the values for the two types of Bayesian audits, standard and RLA.

    Bayesian Bayesian Audit sample size,
    error Audit Type 200 400 800 1,600 3,200 6,400 12,800 25,600 51,200
    0.1 Standard 110 213 419 826 1,636 3,250 6,468 12,889 25,702
    RLA 120 230 443 863 1,691 3,331 6,585 13,049 25,897
    0.05 Standard 112 217 424 833 1,646 3,264 6,487 12,914 25,731
    RLA 122 232 447 868 1,698 3,339 6,596 13,063 25,913
    0.005 Standard 119 226 437 852 1,672 3,300 6,537 12,978 25,804
    RLA 127 239 456 880 1,715 3,363 6,627 13,103 25,957
    Table 3: Values of computed for the Bayesian RLA with and constant , and compared to the Bayesian audit with the same parameters

    Because the range in the values of is very large, the differences between the values of for the two types of audits are not visible in a figure. Instead, Figure 1 contains plots of the difference between for the Bayesian RLA and the standard Bayesian audit.

    Figure 1: The minimum number of winner votes for the Bayesian RLA () less those for the standard Bayesian audit as a function of sample size.
  2. , risk measure 0.001.

    We computed values of for , two candidates and risk measure 0.001 and audit sample sizes from 9-78 and compared the following audits:

    1. Traditional RLA with replacement, . That is, if the declared winner has won the election, we assume it is with a fractional vote count of .

    2. Traditional RLA without replacement, .

    3. Bayesian RLA

      corresponding to the uniform distribution. That is, the prior is uniform over all winning tallies, and the only possibility for

      is a fractional vote of (a tie), with probability 0.5. The fractional vote of in the traditional RLAs was chosen because the center of mass of the Bayesian prior when is a fractional vote of 0.75.

    4. The Bayesian audit corresponding to the uniform distribution.

    Figure 2 plots the values of for samples sizes from 9 through 75. We observe that the audits as listed above are in increasing order of leniency. In particular, we note that the Bayesian RLA requires fewer votes for the winner than does the traditional RLA without replacement, which is interesting. We also notice that the traditional RLA with replacement requires the largest number of votes for the winner, and the Bayesian audit the smallest. This is as expected. Note that traditional RLAs are denoted BRAVO-like RLAs in the figure.

Figure 2: Minimum number of winner votes as a function of sample size

6 Conclusions and Future Work

We have defined a risk-limiting Bayesian polling audit for two-candidate elections and demonstrated that any Bayesian polling audit for two-candidate elections is a simple comparison test between the number of votes for the announced winner in a sample and a pre-computed value for that sample size. Open questions include the application of this model to comparison audits and audits for more complex elections. Also open are the problems of an efficient algorithm to obtain the pre-computed values and the use of this approach in optimizing various election-related criteria.

7 Acknowledgements

The author gratefully acknowledges conversations with Mark Lindeman, Ronald L. Rivest and Philip B. Stark, and some experimentation by Shiva Omrani.


  • [1] M. Lindeman and P. B. Stark. A gentle introduction to risk-limiting audits. IEEE Security & Privacy, 10(5):42–49, 2012.
  • [2] M. Lindeman, P. B. Stark, and V. S. Yates. BRAVO: Ballot-polling risk-limiting audits to verify outcomes. In EVT/WOTE, 2012.
  • [3] K. Ottoboni, P. B. Stark, M. Lindeman, and N. McBurnett. Risk-limiting audits by stratified union-intersection tests of elections (SUITE). In Electronic Voting - Third International Joint Conference, E-Vote-ID 2018, Bregenz, Austria, October 2-5, 2018, Proceedings, pages 174–188, 2018.
  • [4] R. L. Rivest. Clipaudit—a simple post-election risk-limiting audit. arXiv:1701.08312.
  • [5] R. L. Rivest. https://github.com/ron-rivest/2018-bptool.
  • [6] R. L. Rivest and E. Shen. A Bayesian method for auditing elections. In EVT/WOTE, 2012.
  • [7] A. Wald. Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics, 16(2):117–186, 1945.

8 Appendix

Theorem 1: The -Bayesian audit is of the form:

where is the ratio:


Further, , and:

Proof: The audit stops when the estimation error is smaller than ; because this is a binary election, it stops with when:

and, hence, when:

Thus when:

Similarly, when:

Hence the Bayesian Audit is of the form:

where is the ratio: