 # A strong law of large numbers for scrambled net integration

This article provides a strong law of large numbers for integration on digital nets randomized by a nested uniform scramble. This strong law requires a square integrable integrand and it holds along a sequence of sample sizes of the form n=mb^k for m=1,...,M and k>0 and a base b>2.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Randomized quasi-Monte Carlo (RQMC) methods support uncertainty quantification of quasi-Monte Carlo (QMC) estimates of an integral. When the integrand of interest has bounded variation in the sense of Hardy and Krause, then these RQMC methods satisfy both a strong and a weak law of large numbers, respectively SLLN and WLLN. There are however many important cases where the integrand is not of bounded variation. These include integrable singularities

[3, 12, 33] and integrands with kinks and/or jumps [9, 10, 11, 13]. In those cases, we can easily get a WLLN, if the integrand is in . This article proves an SLLN for scrambled net quadrature with integrands in .

To keep this paper at a manageable length, some background in QMC and RQMC is assumed. For QMC, see the survey  or the monographs [25, 6]. For a survey of RQMC methods see .

An outline of this paper is as follows. Section 2 introduces notation, reviews the SLLN and WLLN for Monte Carlo and some quasi-Monte Carlo counterparts. It includes a lemma to show that functions of bounded variation in the sense of Hardy and Krause must also be Riemann integrable. That is either a new result or one hard to find in the literature. Section 3 defines digital nets and describes some properties of randomly scrambled digital nets. Section 4 has the main result. It is a strong law of large numbers for scrambled net sampling. The integrand is assumed to be in . The strong law holds along a set of sample sizes with a geometric spacing. More precisely the given sample sizes are of the form for all and all integers where is the base used to describe the underlying digital nets. This is not a practical hindrance as those include the best sample sizes for digital nets and sequences. For instance, digital nets are designed for equal to a power of . Section 5 provides some additional context and discussion.

## 2 Notation and background

We are given an extended real function for some dimension . The Monte Carlo (MC) method takes independent and estimates by The WLLN is that for any ,

 limn→∞Pr(|^μMCn−μ|>ϵ)=0. (1)

The SLLN is that

 (2)

which we may write as to parallel the WLLN.

In QMC sampling, the

are constructed so that the discrete distribution placing probability

on each of

(with repeated points counted multiple times) is close to the continuous uniform distribution on

. There are various ways, called discrepancies, to quantify the distance between these discrete and continuous measures. The most widely used discrepancy is

 D∗n=D∗n(x1,…,xn)=supa∈[0,1]d∣∣∣1nn∑i=11{xi∈[0,a]}−d∏j=1aj∣∣∣

where .

Deterministic QMC needs no analogue to the WLLN but it does have an SLLN analogue. Let where now the points have been chosen to have small discrepancy. If is Riemann integrable and then [18, p. 3]

 limn→∞^μQMCn=μ (3)

providing the QMC version of the SLLN. There is a converse, where if whenever , then is Riemann integrable. See the references and discussion in . Riemann integrable must also be in .

A better known result about QMC is the Koksma-Hlawka inequality

 |^μQMCn−μ|⩽D∗n×VHK(f) (4)

(see ) where is the total variation of in the sense of Hardy and Krause. If , then we write . We don’t need bounded variation to get an SLLN for QMC. It does however give some information on the rate of convergence because some QMC constructions provide infinite sequences whose initial subsequences satisfy

 D∗n(x1,…,xn)=O(log(n)dn).

The counterpart in MC to the Koksma-Hlawka inequality is that

 E((^μMCn−μ)2)1/2=n−1/2σ(f)

when, for we have . Where the rate for QMC comes after restricting from Riemann integrability to bounded variation, the rate for MC comes about after restricting from to . The MC counterpart is exact while the QMC version is an upper bound.

A Riemann integrable function is not necessarily in BVHK. For instance is Riemann integrable but, for , it is not in BVHK . A function in BVHK is necessarily Riemann integrable. This result is hard to find in the literature. It must have been known to Hardy, Krause, Hobson and others over a century ago, at least for , which earlier work emphasized. Here is a proof.

###### Lemma 1.

If , then is also Riemann integrable.

###### Proof.

If is in then where are uniquely determined completely monotone functions on with [1, Theorem 2]. Completely monotone functions are, a fortiori, monotone. Now both are bounded monotone functions on . They are then Riemann integrable by the Corollary in . ∎

While QMC has a superior convergence rate to MC for , MC has an advantage over QMC in that is simple to estimate from independent replicates, while is extremely difficult to estimate  and much harder to estimate than

. In a setting where attaining accuracy is important it will also be important to estimate the attained accuracy. Independent replication of RQMC estimates allows variance estimation for them

[28, 20].

In RQMC, one starts with points having a small discrepancy and randomizes them producing points . These points satisfy the following conditions: individually , and collectively, have low discrepancy. The RQMC estimate of is . From the uniformity of the points we find that .

## 3 Scrambled nets and sequences

In this section, we focus on scrambled versions of digital nets and sequences. Let be an integer base. Let and where . Then

 E(k,c)=d∏j=1[cjbkj,cj+1bkj) (5)

is an elementary interval in base . It has volume where .

###### Definition 1.

For integers , and , the points for are a -net in base if

 n∑i=11{xi∈E(k,c)}=bm−|k|

holds for every elementary interval from (5) with .

An elementary interval of volume should ideally contain points from . In a digital net, every elementary interval that should ideally contain of the points does so. For any given , and , smaller imply finer equidistribution. It is not always possible to attain .

###### Definition 2.

For integers , and , the points for are a -sequence in base if every subsequence of the form for integers and is a -net in base .

The best known values of for nets and sequences are recorded in the online resource MinT described in , which also includes lower bounds. The Sobol’ sequences of  are -sequences in base . There are newer versions of Sobol’s sequence with improved ‘direction numbers’ in [17, 40]. The Faure sequences  have but require that the base be a prime number . Faure’s construction was generalized to prime powers . The best values of for base are in the Niederreiter-Xing sequences of [26, 27].

Randomizations of digital nets and sequences operate by applying certain random permutations to their base expansions. For details see the survey in . We will consider the ‘nested uniform’ scramble from .

If is a -net in base then after applying a nested uniform scramble, the resulting points are a -net in base with probabilty one . If for are a -sequence in base then after applying a nested uniform scramble, the resulting points for are a -sequence in base with probability one .

If and is based on a nested uniform scramble of a -sequence in base with sample sizes for integers , then as . For smooth enough , for any [29, 34].

The main result that we will use is as follows. Let and write for the variance of for . Then for a scrambled -net in base we have

 E((^μRQMCn−μ)2)⩽Γσ2n (6)

for some [30, Theorem 1]. The RQMC estimate for these scrambled nets cannot have more than times the mean squared error that MC has. The value of is found using some conservative upper bounds. We can use . If this can be reduced to , and for it can be reduced to . The quantity arises as an upper bound on an infinite set of ‘gain coefficients’ relating the RQMC variance to the MC variance for parts of a basis expansion of .

## 4 RQMC laws of large numbers

If , then there is an SLLN for RQMC from the Koksma-Hlawka inequality (4) because the RQMC points have . The same argument handles the more general case of Riemann integrable .

###### Theorem 1.

Let be Riemann integrable. Let for be RQMC points with . Then

 Pr(limn→∞^μRQMCn=μ)=1.
###### Proof.

On the event , which has probability one, we have by (3). ∎

The usual results for RQMC show that as for . From that a WLLN follows by Chebychev’s inequality.

An SLLN is more complicated. We will provide one for points taken from a scrambled digital sequence using a subset of sample sizes. For integers and , let

 N=N(b,M)={mbk∣m∈{1,…,M},k∈N}.

Denote the unique elements of by for with . The used in sample sizes should not be confused with the used in the definition of nets.

Restricting the sample sizes is not a serious impediment to using RQMC. It is actually a best practice when the point sets are designed for sample sizes equal to a power of a prime number. Sobol’  recommends considering sample sizes in a geometric progression such as , not an arithmetic one. One of his points is that when errors smaller than are possible they cannot be expected to occur at consecutive sample sizes and for then the ’st point on its own would be unrealistically good. See  for more details on how rate optimal sample sizes for equal weight rules must be geometrically spaced. The empirical results for scrambled Faure sequences in  were noticeably better for sample sizes . Scrambled net variances are not necessarily monotone in . For instance, with for scrambled van der Corput points (a -sequence in base ) have variance for even

but not for odd

.

###### Theorem 2.

Let be a -sequence in base , with gain coefficients no larger than and randomized as in . Let with and . Then

for any set of sample sizes of the form .

###### Proof.

For and let . For we have

 ^μnℓ=1mm∑r=1^μr,k.

Because RQMC is unbiased, we have . Next, define

 τ2r,k=var(^μr,k)⩽Γσ2bk.

Then because the correlations among cannot be larger than one,

 var(^μnℓ)⩽1m2m∑r=1m∑s=1Γσ2bk=Γσ2bk=mΓσ2nℓ. (7)

For any , Chebychev’s inequality yields

 Pr(|^μnℓ−μ|>ϵ)⩽mΓσ2ϵ2nℓ.

Then

 ∞∑ℓ=1Pr(|^μnℓ−μ|>ϵ)⩽∞∑ℓ=1mΓσ2ϵ2nℓ⩽∞∑k=0M∑m=1mΓσ2ϵ2(mbk)=bb−1MΓσ2ϵ2.

The second inequality above is not necessarily an equality because summing over and may count some more than once. Because that sum is finite, by the Borel-Cantelli lemma. Therefore . ∎

The set of sample sizes could be made even larger by allowing to depend on . It would suffice to have . For instance for some or even would still yield an SLLN.

A -sequence in base is extensible in that after using points of the sequence we can obtain an point rule by simply adjoining the next points in the sequence. We can get an SLLN for a non-extensible “triangular array” construction where the points in the rule with are not necessarily also used for for . For let be the union of digital nets, each with gain coefficient , and given a nested uniform scramble. Then the proof in Theorem 2 will go through so long as

 ∞∑ℓ=1ΓℓMℓnℓ<∞.

## 5 Discussion

The strong law for averages over nested uniform scrambling of digital nets makes two restrictions not needed in the classical strong law for averages of independent and identically distributed random variables. It requires

instead of just , and it uses a strict subset of sample sizes instead of all . The SLLN in Theorem 2 applies to a sufficiently large collection of sample sizes for applications. It would be useful to weaken the assumption that . Even a WLLN for for some , especially , would be valuable.

There are other ways to scramble digital nets and sequences. The linear scrambles of  require less space than the nested uniform scramble. They have the same mean squared discrepancy as the nested uniform scramble  and so they might also satisfy an SLLN. A digital shift [20, 31] does not produce the same variance as the nested uniform scramble and it does not satisfy the critically important bound (6

) on gain coefficients, so the methods used here would not provide an SLLN for it. The nested uniform scramble is the only one for which central limit theorems have been proved

[21, 2].

A second major family of RQMC methods has been constructed from lattice rules . Points on a lattice in are randomized into , for . That is, they are shifted with wraparound in what is known as a Cranley-Patterson rotation . For an extensible version of shifted lattice rules, see . The Cranley-Patterson rotation does not provide a bound like (6) because there are functions with , and so a proof of an SLLN for this form of RQMC would require a different approach. The fact that is possible does not provide a counter-example to an SLLN because this equality might only hold for a finite number of in the infinite sequence. Given a class of functions with for all , all , and some , we get an SLLN for if . Some such bounds for randomly shifted lattices appear in  though they hold for specific not necessarily an infinite sequence of them.

## Acknowledgments

I thank Max Balandat for posing the problem of finding and SLLN for randomized QMC. This work was supported by grant IIS-1837931 from the U.S. National Science Foundation.

## References

•  Ch. Aistleitner and J. Dick. Functions of bounded variation, signed measures, and a general Koksma-Hlawka inequality. Acta Arithmetica, 167(2):143–171, 2015.
•  K. Basu and R. Mukherjee. Asymptotic normality of scrambled geometric net quadrature. The Annals of Statistics, 45(4):1759–1788, 2017.
•  K. Basu and A. B. Owen. Quasi-Monte Carlo for an integrand with a singularity along a diagonal in the square. In J. Dick, F. Y. Kuo, and H. Woźniakowski, editors, Contemporary Computational Mathematics-A Celebration of the 80th Birthday of Ian Sloan, pages 119–130. Springer, 2018.
•  R. Cranley and T.N.L. Patterson. Randomization of number theoretic methods for multiple integration. SIAM Journal of Numerical Analysis, 13:904–914, 1976.
•  J. Dick, F. Y. Kuo, and I. H. Sloan. High-dimensional integration: the quasi-Monte Carlo way. Acta Numerica, 22:133–288, 2013.
•  J. Dick and F. Pillichshammer. Digital sequences, discrepancy and quasi-Monte Carlo integration. Cambridge University Press, Cambridge, 2010.
•  C. Doerr, M. Gnewuch, and M. Wahlström. Calculation of discrepancy measures and applications. In Chen W., Srivastav A., and Travaglini G., editors, A Panorama of Discrepancy Theory, pages 621–678. Springer, 2014.
•  H. Faure. Discrépance de suites associées à un système de numération (en dimension ). Acta Arithmetica, 41:337–351, 1982.
•  M. Griebel, F. Y. Kuo, and I. H. Sloan. The smoothing effect of integration in and the ANOVA decomposition. Mathematics of Computation, 82(281):383–400, 2013.
•  M. Griebel, F. Y. Kuo, and I. H. Sloan. Note on “The smoothing effect of integration in and the ANOVA decomposition”. Mathematics of Computation, 86(306):1847–1854, 2017.
•  A. Griewank, F. Y. Kuo, H. Leövey, and I. H. Sloan. High dimensional integration of kinks and jumps–Smoothing by preintegration. Journal of Computational and Applied Mathematics, 344:259–274, 2018.
•  J. Hartinger and R. Kainhofer. Non-uniform low-discrepancy sequence generation and integration of singular integrands. In H. Niederreiter and D. Talay, editors, Monte Carlo and Quasi-Monte Carlo Methods 2004, pages 163–179. Springer, 2006.
•  Z. He and X. Wang. On the convergence rate of randomized quasi–Monte Carlo for discontinuous functions. SIAM Journal on Numerical Analysis, 53(5):2488–2503, 2015.
•  F. J. Hickernell. Koksma-Hlawka inequality. Wiley StatsRef: Statistics Reference Online, 2014.
•  F. J. Hickernell, H. S. Hong, P. L’Ecuyer, and C. Lemieux. Extensible lattice sequences for quasi-Monte Carlo quadrature. SIAM Journal on Scientific Computing, 22(3):1117–1138, 2000.
•  F. J. Hickernell and R. X. Yue. The mean square discrepancy of scrambled -sequences. SIAM Journal of Numerical Analysis, 38:1089–1112, 2000.
•  S. Joe and F. Y. Kuo. Constructing Sobol’ sequences with better two-dimensional projections. SIAM Journal on Scientific Computing, 30(5):2635–2654, 2008.
•  L. Kuipers and H. Niederreiter. Uniform distribution of sequences. Wiley, New York, 1974.
•  Boris Lavrič. Continuity of monotone functions. Archivum Mathematicum, 29(1):1–4, 1993.
•  P. L’Ecuyer and C. Lemieux. A survey of randomized quasi-Monte Carlo methods. In M. Dror, P. L’Ecuyer, and F. Szidarovszki, editors, Modeling Uncertainty: An Examination of Stochastic Theory, Methods, and Applications, pages 419–474. Kluwer Academic Publishers, 2002.
•  W.-L. Loh. On the asymptotic distribution of scrambled net quadrature. Annals of Statistics, 31(4):1282–1324, 2003.
•  J. Matoušek. On the L–discrepancy for anchored boxes. Journal of Complexity, 14:527–556, 1998.
•  H. Niederreiter. Pseudo-random numbers and optimal coefficients. Advances in Mathematics, 26:99–181, 1977.
•  H. Niederreiter. Point sets and sequences with small discrepancy. Monatshefte fur mathematik, 104:273–337, 1987.
•  H. Niederreiter. Random Number Generation and Quasi-Monte Carlo Methods. S.I.A.M., Philadelphia, PA, 1992.
•  H. Niederreiter and C. Xing. Low-discrepancy sequences and global function fields with many rational places. Finite Fields and Their Applications, 2:241–273, 1996.
•  H. Niederreiter and C. Xing. Quasirandom points and global function fields. In S. Cohen and H. Niederreiter, editors, Finite Fields and Applications, volume 233, pages 269–296, Cambridge, 1996. Cambridge University Press.
•  A. B. Owen. Randomly permuted -nets and -sequences. In H. Niederreiter and P. Jau-Shyong Shiue, editors, Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, pages 299–317, New York, 1995. Springer-Verlag.
•  A. B. Owen. Scrambled net variance for integrals of smooth functions. Annals of Statistics, 25(4):1541–1562, 1997.
•  A. B. Owen. Scrambling Sobol’ and Niederreiter-Xing points. Journal of Complexity, 14(4):466–489, December 1998.
•  A. B. Owen. Variance with alternative scramblings of digital nets. ACM Transactions on Modeling and Computer Simulation, 13(4):363–378, 2003.
•  A. B. Owen. Multidimensional variation for quasi-Monte Carlo. In J. Fan and G. Li, editors, International Conference on Statistics in honour of Professor Kai-Tai Fang’s 65th birthday, 2005.
•  A. B. Owen. Randomized QMC and point singularities. In H. Niederreiter and D. Talay, editors, Monte Carlo and Quasi-Monte Carlo Methods 2004, pages 403–418. Springer, 2006.
•  A. B. Owen. Local antithetic sampling with scrambled nets. Annals of Statistics, 36(5):2319–2343, 2008.
•  A. B. Owen. A constraint on extensible quadrature rules. Numerische Mathematik, 132(3):511–518, 2016.
•  R. Schürer and W. Ch. Schmid. MinT: new features and new results. In P. L’Ecuyer and A. B. Owen, editors, Monte Carlo and Quasi-Monte Carlo Methods 2008. Springer, 2009.
•  I. H. Sloan and S. Joe. Lattice Methods for Multiple Integration. Oxford Science Publications, Oxford, 1994.
•  I. M. Sobol’. The distribution of points in a cube and the accurate evaluation of integrals (in Russian). Zh. Vychisl. Mat. i Mat. Phys., 7:784–802, 1967.
•  I. M. Sobol’. On quasi-Monte Carlo integrations. Mathematics and Computers in Simulation, 47:103–112, 1998.
•  I. M. Sobol’, D. Asotsky, A. Kreinin, and S. Kucherenko. Construction and comparison of high-dimensional Sobol’ generators. Wilmott magazine, 2011(56):64–79, 2011.