# Entropic Compressibility of Lévy Processes

In contrast to their seemingly simple and shared structure of independence and stationarity, Lévy processes exhibit a wide variety of behaviors, from the self-similar Wiener process to piecewise-constant compound Poisson processes. Inspired by the recent paper of Ghourchian, Amini, and Gohari, we characterize their compressibility by studying the entropy of their double discretization (both in time and amplitude) in the regime of vanishing discretization steps. For a Lévy process with absolutely continuous marginals, this reduces to understanding the asymptotics of the differential entropy of its marginals at small times. We generalize known results for stable processes to the non-stable case, and conceptualize a new compressibility hierarchy of Lévy processes, captured by their Blumenthal-Getoor index.

## Authors

• 7 publications
• 8 publications
• 7 publications
• ### Estimation of the multifractional function and the stability index of linear multifractional stable processes

In this paper we are interested in multifractional stable processes wher...
11/22/2017 ∙ by Thi To Nhu Dang, et al. ∙ 0

• ### Representations of Hermite processes using local time of intersecting stationary stable regenerative sets

Hermite processes are a class of self-similar processes with stationary ...
10/16/2019 ∙ by Shuyang Bai, et al. ∙ 0

• ### Exact Simulation of the Extrema of Stable Processes

We exhibit an exact simulation algorithm for the supremum of a stable pr...
06/05/2018 ∙ by Jorge Ignacio González Cázares, et al. ∙ 0

• ### ε-strong simulation of the convex minorants of stable processes and meanders

Using marked Dirichlet processes we characterise the law of the convex m...
10/29/2019 ∙ by Jorge Ignacio González Cázares, et al. ∙ 0

• ### A Dynamic Process Reference Model for Sparse Networks with Reciprocity

Many social and other networks exhibit stable size scaling relationships...
12/31/2019 ∙ by Carter T. Butts, et al. ∙ 0

• ### Posterior Convergence of α-Stable Sheets

This paper is concerned with the theoretical understanding of α-stable s...
07/06/2019 ∙ by Neil K. Chada, et al. ∙ 0

• ### Posterior Convergence Analysis of α-Stable Sheets

This paper is concerned with the theoretical understanding of α-stable s...
07/06/2019 ∙ by Neil K. Chada, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Lévy processes generalize the Wiener process by relaxing the Gaussianity of increments, while retaining their stationarity and independence. They have proven to be a fruitful and flexible stochastic continuous-domain model in many applications, including financial mathematics [66], movement patterns of animals [40], turbulence [4], and sparse signal processing [74], among others.

In this paper, we follow an entropy-based approach initiated in [32] to quantify the compressibility of a Lévy process, assuming that the process is approximated by uniform sampling in time and uniform quantization in amplitude. Specifically, let be a Lévy process and for

, consider the vector

obtained by sampling over the interval with a period of . Since the increments of are independent and stationary, it is more convenient to work with the vector of independent and identically distributed increments where for (and by definition). For an integer and , denote by , the quantization of with step size . Then, we define the entropy of the Lévy process for sampling period and quantization step as

 Hn,m(L)=H([Δ1]m,…,[Δn]m) (1)

where

denotes the entropy of the discrete random variable

.

The main goal of this paper is to understand the asymptotic behavior of (1) when . Due to the characterization of entropy in terms of compressibility, a slower asymptotic growth of implies that fewer bits are required to encode an approximation of of a given quality. Therefore, the asymptotic behavior of is a measure of the entropic compressibility of the Lévy process .

### 1.1 Contributions: the Asymptotic Entropy of Lévy Processes

Our work is directly inspired by the contribution of Hamid Ghourchian, Arash Amini, and Amin Gohari in [32], who introduced the quantity —with a slightly different but equivalent perspective—and characterized its asymptotic behavior for important classes of Lévy processes, mainly the compound Poisson and stable processes.

We provide a general analysis of the entropy of a Lévy process , with a particular focus on Lévy processes whose marginals are absolutely continuous, thus allowing us to reduce this question to the study of the differential entropy at small times . This reduction mostly benefits from the seminal contributions of [32], to which we bring some useful complements. Our main results, detailed below, extend the ones of [32] to all (possibly non-stable) Lévy processes whose marginals are absolutely continuous.

• [label=]

• The entropy of locally self-similar Lévy processes. A Lévy process is locally symmetric and self-similar if its rescaled versions converge locally to a symmetric non-trivial random process (see Definition 4.1). The rescaling depends on the local self-similarity order of the Lévy process, that is, the self-similarity order of its local limit, which is itself related to its Blumenthal–Getoor index (see Section 2.3). We show in Theorem 4.3 that the entropy of a locally symmetric and self-similar Lévy process has the same asymptotic behavior as the symmetric--stable process with ; that is,

 h(Lt)∼t→0+1βlogt. (2)
• An Upper-Bound for the Entropy of Lévy Processes. For the general case, we obtain an upper-bound on the entropy in terms of the Blumenthal–Getoor index of the Lévy process. More precisely, we show that

 limsupt→0+h(Lt)log(1/t)≤−1β. (3)

A direct consequence of this result is that for Lévy processes with , the entropy diverges super-logarithmically as , making them more compressible than any locally symmetric and self-similar Lévy process.

These two results together reveal a new entropy-based compressibility hierarchy of Lévy processes determined by their Bluementhal–Getoor index. Among Lévy processes with absolutely continuous marginals, the ones with Blumenthal–Getoor index are the most compressible, and more generally the smaller , the more compressible . We moreover exemplify our main contributions on several classes of Lévy processes.

### 1.2 Related Work

##### Entropy & Compressibility.

The notion of entropy introduced in 1948 by Claude Shannon in two successive groundbreaking papers [67, 68] is a fundamental measure of the quantify of information of discrete random sources and exactly captures their compressibility as formalized by the source coding theorem [15, Theorem 5.4.1]

. Initially defined for discrete random variables, it can be generalized to continuous random variables as the differential entropy, whose relation to discrete entropy via quantization was studied by Alfréd Rényi

[57]. Note however that contrary to the discrete entropy, there is no universal notion for quantifying the information of continuous random sources and the differential entropy is only one of several generalizations, with its strengths and weaknesses [43].

Beyond discrete random vectors, one can consider the entropy of random sequences, defined as the averaged limit of the vector case [15, Section 4.2]. Going one step further, several generalizations to continuous-time random processes have been proposed, as discussed extensively in the monograph of Shunsuke lhara [41]. In this paper, we follow the recent work of Ghourchian et al. [32], which provides a new definition of the entropy of a random process. Their approach is based on a double discretization in time (sampling) and amplitude (quantization), corresponding to (1) for Lévy processes and relying on discrete entropy. While these authors consider general Lévy processes, whose marginals are not necessarily absolutely continuous, we note that among Lévy processes with absolutely continuous marginals, they only quantify the entropy of stable processes.

##### Sparsity & Lévy Processes.

A signal of interest is sparse if it admits a concise representation that captures most of its information. Many naturally-occurring signals are deeply structured and admit such sparse representations, calling for sparse models and sparsity-promoting methods in the analysis and synthesis of signals. This led for example to the introduction of methods in statistical learning [73, 37] which are ubiquitous in the field of compressed sensing  [19, 13].

Continuous-domain signals present their own challenges since adequate representations should ideally capture the fine details of possibly fractal-type sample paths [51]. It is well-known that classic Gaussian models fail at modelling sparsity [72, 53, 24]

, in the sense that they generate poorly compressible data. Stable models with infnite variance

[61, 54] and more generally Lévy processes and Lévy fields [74] have been proposed beyond Gaussian models. Those are particularly interesting since they include a wide range of random processes from non-sparse models, such as the Wiener process, to very sparse models, such as compound Poisson processes, whose rate of innovation is finite [77]. There is strong empirical evidence that Lévy processes are useful in modeling sparsity in signal processing [74]. To the best of our knowledge, [32] complemented by the present paper is the first attempt to provide an information-theoretic justification of this observation.

##### Approximation-theoretic Compressibility.

We also mention a distinct area of research which aims at quantifying the sparsity of random models, with a different perspective based on the theory of approximation. It was initiated with the study of independent and identically distributed random sequences [14] whose compressibility is measured by the asymptotic behavior of truncated subsequences [1] and is strongly linked to the heavy-tailedness of the distribution of the sequence [36, 70]. Extensions to stationary and ergodic random sequences have also been proposed [69].

A continuous-domain function is compressible if most of its information is captured by a few coefficients in an adequate dictionary. In this regard, wavelet representations are well-known for their excellent compression rate [50]. A natural line of research, has therefore been to study the wavelet approximability of a random model, understood as the convergence rate of its best approximation in wavelet bases. It was shown in a series of work [27, 25, 3], that the approximation error of the best -term approximation of a Lévy process in wavelet bases behaves asymptotically like , where is the Blumenthal–Getoor of the process, and decays faster than any polynomial for . This shows a wavelet-based hierarchy of Lévy processes, from the less sparse (the Wiener process) to the sparsest (compound Poisson or Lévy process [28]. Remarkably, this coincides with the information-theoretic compressibility hierarchy provided in this paper.

### 1.3 Outline

Section 2 introduces the family of Lévy processes and their Blumenthal–Getoor index. The entropy of a Lévy process is rigorously defined in Section 3, where we also provide existence results. Section 4 contains the main results of the paper: we first characterize the asymptotic behavior of the entropy of locally symmetric and self-similar Lévy processes in Section 4.2, and then deduce a general upper bound for the entropy of any Lévy process in Section 4.3. We apply our theoretical findings to specific classes of Lévy processes by characterizing their small-time entropic behavior in Section 5. Finally, a summary and discussion of our results is provided in Section 6.

## 2 Lévy Processes and their Blumenthal–Getoor Index

We introduce the family of Lévy processes in Section 2.1 and the subfamily of symmetric stable processes in Section 2.2. As we shall see in Sections 5 and 4, the entropic compressibility of a Lévy process is captured by its Blumenthal–Getoor index, whose definition and main properties are recalled in Section 2.3.

### 2.1 Lévy Processes

Lévy processes are named after Paul Lévy, who popularized their study in 1937 [47, Chapter VII]111Lévy processes had been previously introduced by Bruno de Finetti [31]. Paul Lévy generalized existing results to the class of additive processes, that is, processes satisfying Definition 2.1 with the exception of stationarity. We refer the interested reader to [49] for a detailed historical exposition of this matter.. They generalize the Wiener process by relaxing the requirement that increments be Gaussian. A brief presentation is given thereafter; more details can be found in the classic monographs [2, 7, 62].

###### Definition 2.1.

A Lévy process is a continuous-time random process satisfying:

1. almost surely (a.s.).

2. Stationary increments: for all , and have the same law.

3. Independent increments: for all and all , the random variables are mutually independent.

4. Sample paths regularity: is a.s. right-continuous with left limits over .

Recall that a random variable is infinitely divisible if it can be decomposed as the sum of i.i.d. random variables for all  [62]. Lévy processes are intimately linked to infinitely divisible random variables [62, Section 7]. In particular, the marginals are infinitely divisible for all . Indeed, we can write for each and ,

 Lt=n∑k=1(Lknt−Lk−1nt), (4)

where is i.i.d. since has independent and stationary increments.

Moreover, the law of a Lévy process is completely characterized by the infinitely divisible law of , its marginal at time

of can be expressed as , for , where , the characteristic exponent of , is a continuous function admitting a Lévy–Khintchine representation [62, Theorem 8.1]

 Ψ(ξ)=iμξ−σ2ξ22+∫R(eiξt−1−iξt1|t|≤1)ν(dt),∀ξ∈R, (5)

with , , and is a Lévy measure. The latter means that is a non-negative measure on such that

 ∫|t|≤1t2dν(t)<∞,∫|t|>1dν(t)<∞,andν({0})=0.

We call the Lévy triplet of , which uniquely determines . Then, for all , the characteristic function of the infinitely divisible random variable is

 ΦLt(ξ)=exp(tΨ(ξ)),∀ξ∈R.

### 2.2 Symmetric-α-Stable Lévy Processes

Among Lévy processes, we now define the subfamily of symmetric--stable processes, which will play an important role in our study, as potential limits in law of rescaled Lévy processes (see Section 4.1). More information can be found in the classical monograph by Gennady Samorodnitsky and Murad Taqqu [61].

A random variable is stable if, for all , there exists and such that

 X1+⋯+Xn(L)=cnX+dn, (6)

where the are independent copies of and stands for equality in law. From the definition, we readily see that a stable random variable is infinitely divisible. The entire family of stable random variables is described by four parameters [34, Section 34]. If is moreover symmetric, then its characteristic function takes the form

 ΦX(ξ)=exp(−γ|ξ|α),∀ξ∈R,

where and . In this case, Eq. (6) is satisfied with and  [30, Theorem 1, Section VI.1]. The parameter is simply a scale parameter, while the parameter deeply influences the properties of . This justifies the terminology symmetric--stable laws. For instance, the

th moment

of is finite if and only if (in which case is a Gaussian random variable) or  [61, Proposition 1.2.16].

For , a symmetric--stable Lévy process (or SS process) is a Lévy process for which is a SS random variable. The characteristic exponent of is thus given by

 Ψ(ξ)=−γ|ξ|α,∀ξ∈R (7)

and, for each , is SS with characteristic function .

### 2.3 The Blumenthal–Getoor Index of Lévy Processes

The Blumenthal–Getoor index was introduced by Robert M. Blumenthal and Ronald K. Getoor in 1961 to characterize the small-time behavior of Lévy processes [8]. Since then, it has been recognized as a key quantity to characterize the local Besov regularity [64, 63, 3] or the variations [59] of the sample paths, the Hausdorff dimension of the image set [42, 20]

, moment estimates

[48, 18, 25, 46], the local self-similarity [26], or the local wavelet compressibility [28] of Lévy processes and their generalizations. We demonstrate in this paper that it also quantifies the asymptotic behavior of the entropy of Lévy processes.

A Lévy process satisfies the sector condition if there exists a constant such that

 |IΨ(ξ)|≤C|RΨ(ξ)|,∀ξ∈R, (8)

with its characteristic exponent, and where and respectively stand for the real part and imaginary part of complex numbers. This condition ensures that no drift is dominating the process. It typically excludes the pure drift (deterministic) process with , for which , and is automatically satisfied by symmetric Lévy processes, for which is real. See [11] for additional discussions about the sector condition. We shall always assume that the sector condition is satisfied in this paper without further mention.

###### Definition 2.2.

The Blumenthal–Getoor index of a Lévy process with characteristic exponent is defined by

 β=inf{p>0,|Ψ(ξ)||ξ|p⟶|ξ|→∞0}. (9)

The Blumenthal–Getoor index lies in since the characteristic exponent is asymptotically dominated by  [22, Proposition 2.4]. The characteristic exponent of a SS process being given by (7), we deduce that its index is . The characteristic exponent of the Laplace process, for which is a Laplace random variable, is given by  [44]. We therefore have in this case. This is also the case for the gamma process (see Section 5.2).

###### Remark.

It comes as no surprise that the local behavior of a Lévy process is captured (via the index ) by the asymptotic behavior of its characteristic exponent. The latter is indeed the logarithm of the characteristic function of the law of

, whose local properties are known to be linked to asymptotic properties of its Fourier transform.

## 3 Entropy of Lévy Processes

### 3.1 Quantization and Entropy of Random Variables

We briefly review the definitions of discrete and differential entropy. The former applies to random variables taking values in a countable space, and the latter applies to absolutely continuous random variables, i.e.

, variables whose distribution is absolutely continuous with respect to the Lebesgue measure and thus admit a probability density function.

###### Definition 3.1.

Let be a discrete random variable taking values into a countable space . The discrete entropy of is defined by

 (10)

with the usual convention . Since the summand in (10) is non-negative, is either or a non-negative real. In the latter case, we say that has finite entropy.

Let

be an absolutely continuous real-valued random variable with probability density function

. The differential entropy of is

 h(X)=−∫RpX(x)logpX(x)dx=−E[logpX(X)], (11)

which is well-defined in provided that either the positive or negative part of the integrand in (11) is integrable. We say that the real random variable has finite differential entropy when it is absolutely continuous and the integrand in (11) is absolutely integrable.

The quantization of order (or -quantization) of a real is defined by

 [x]m=1m⌊mx+1/2⌋,

that is, is the unique element of such that . For a real random variable , the study of the entropy of its quantization , and its relation to (when it exists) was initiated by Alfréd Rényi in [57]. In particular, he established the following result (see also [32, Corollary 1]).

###### Proposition 3.2 ([57, (11) and Thm. 1]).

Let be a real random variable. If , then for all . If furthermore has finite differential entropy, then

 H([X]m)=logm+h(X)+om(1). (12)
###### Remark.

Finiteness of the differential entropy does not necessary imply finiteness of , as exemplified by Rényi in the remark following [57, Theorem 1].

For an absolutely continuous random variable , it is known that finiteness of the log-moment implies that (see e.g. [58, Proposition 1]

). This is proved by a direct application of Gibbs’ inequality to the Kullback–Leibler divergence

for a Cauchy variable . A simple adaption of the proof shows that this condition also implies . We thus obtain the following proposition, proved in Appendix A.

###### Proposition 3.3.

Let be a real random variable such that , then is finite. If moreover is absolutely continuous with bounded probability density function (e.g., if is integrable) then has finite differential entropy and (12) holds.

### 3.2 Definition and Existence of the Entropy of Lévy Processes

The notion of -quantization is extended to finite-dimensional vectors by writing for . Next, we give the definition of the entropy of a Lévy process. Although the presentation is slightly different, our definition is equivalent to the one of Ghourchian et al. [32, Eq. (6)]

, given for a Lévy white noise, which is the weak derivative of a Lévy process.

###### Definition 3.4.

Let be a Lévy process and be integers. First, let

 Xn(L)=(L1/n,L2/n−L1/n,…,L1−L(n−1)/n)

be the random vector of sampled values of . Then, the entropy of with time-quantization and amplitude-quantization is defined as

 (13)

where the second equality is because the coordinates of are independent and distributed as by definition of a Lévy process.

Intuitively, measures the expected length of the most efficient encoding of over , after approximating it both temporally (via sampling) and in amplitude (via quantization). In [32], the authors argue that the asymptotic behavior of as and can be used as a measure of the compressibility of .

A direct corollary of Proposition 3.2 is the following result, implicitly stated in [32].

###### Corollary 3.5 (implicit in [32]).

Let be a Lévy process. If then the entropy is finite for all . If moreover has finite differential entropy for all , then

 (14)
###### Proof.

According to [32, Section IV-B, proof of item 2], implies for all . We conclude from (13) after applying Proposition 3.2 to . ∎

###### Remark.

It is possible to construct Lévy processes whose entropy is infinite, even when the marginals are absolutely continuous for . For this, it suffices to construct such that . An example is provided in Proposition A.1 in Appendix A.

From Proposition 3.3 we obtain the following corollary, proved in Appendix A, showing that finiteness of as well as (14) hold under mild assumptions.

###### Corollary 3.6.

Let be a Lévy process such that . Then, the entropy is finite for all . If moreover the characteristic function is integrable for all , then has finite differential entropy for all and (14) holds.

###### Remark.

Finiteness of the log-moment is a condition which also appears when studying Lévy-driven CARMA processes, for which it is equivalent to the existence of a stationary solution [12, 6]. Note that this condition is in particular implied by the finiteness of the absolute moment for some , which is an equivalent characterization of tempered (i.e., bounded by a polynomial) Lévy processes [23, 17].

Since all Lévy processes considered in this paper have marginals with finite differential entropy, we will henceforth focus on characterizing the asymptotic behavior of as . By (14), any such characterization immediately carries over to the asymptotic behavior of , thus quantifying the compressibility of . We start with the following observations, valid for all Lévy processes with finite differential entropy, and which will be refined in subsequent sections by considering the Blumenthal–Getoor index.

###### Proposition 3.7.

Let be a Lévy process such that has finite differential entropy for all , then:

1. the function is non-decreasing,

2. for .

###### Proof.
1. Recall that whenever and are independent random variables with finite differential entropy. Since has independent increments we obtain for that .

2. For mutually independent variables with finite differential entropy, we have by the entropy-power inequality (see e.g., [76, Eq. (2)]). Applying this inequality to (4) with , for which , we obtain for all that

 2h(L1)≥logn+2h(L1/n). (15)

Consider , and define so that . Then,

 h(Lt)≤h(L1/n)≤h(L1)−12logn≤h(L1)+12logt1−t,

where the first inequality follows from (i), the second from (15), and the last is due to by definition of .∎

###### Remark.

The asymptotic behavior of the upper bound in (ii) is as . Since for a Wiener process , the bound in (ii) is asymptotically tight for general Lévy processes. In Section 4, we will obtain the tighter bound , for Lévy processes with Blumenthal–Getoor index .

## 4 Entropy of Lévy Processes at Small Times

As per the discussion below Corollary 3.6, the entropic compressibility of a Lévy process with finite differential entropy is governed by the small-time asymptotics of , that we quantify in this section in terms of the Blumenthal–Getoor index . We first consider the case of locally symmetric and self-similar Lévy processes, defined in Section 4.1, for which we show in Section 4.2 that when . We then consider general Lévy processes in Section 4.3, for which we obtain an upper bound on . As an important consequence, we deduce that when , diverges super-logarithmically when .

### 4.1 Locally Self-Similar Lévy Processes

###### Definition 4.1.

We say that a random process is

• [label=]

• self-similar of order if the rescaled random process and have the same probability law for all ;

• locally self-similar (LS) of order if converges in law to a non-trivial random process when . If moreover the limiting process is symmetric, we say that is locally symmetric and self-similar (LSS).

Self-similar processes have been studied extensively [71, 21] and include the fractional Brownian motion [52] and generalizations of SS processes [61, Chapter 7]. Locally self-similar processes were introduced in [26, Definition 4.3]. Higher values of in corresponds to zooming in the process at the origin. The limiting process in Definition 4.1 is known to be self-similar of order  [26, Proposition 4.4], which explains the terminology.

The following result, proved in Appendix B, characterizes symmetric self-similar and locally symmetric and self-similar random processes among the family of Lévy processes.

###### Proposition 4.2.

A Lévy process is symmetric and self-similar if and only if it is a SS process. In this case, the self-similarity order is . A Lévy process with Blumenthal–Getoor index is LSS if and only if and its characteristic exponent satisfies

 Ψ(ξ)∼|ξ|→∞−γ|ξ|β (16)

for some constant . The local self-similarity order is then and the limiting process is SS with .

###### Remark.

A locally symmetric self-similar process is not necessarily symmetric itself, but its potential skewness vanishes at small times. Proposition

4.2 can be extended to the non-symmetric case, but we will not cover it in this paper (see Section 6 for a short discussion regarding this point).

###### Remark.

According to Proposition 4.2, the Blumenthal–Getoor index of a LSS Lévy process is necessarily positive. In fact, a Lévy process with is such that converges in law to for all when  [26, Proposition 4.7].

### 4.2 A Small-time Limit Theorem for the Entropy of LSS Lévy Processes

In this section, we consider the differential entropy of an LSS Lévy process , for which we obtain an exact asymptotic expansion as up to a vanishing term.

###### Theorem 4.3.

Let be a LSS Lévy process with Blumenthal–Getoor index and such that has a finite absolute moment for some . Then, the differential entropy is finite for all and

 h(Lt)=1βlogt+h(Yβ)+ot(1), (17)

where is a SS random variable with and when .

We shall use the following lemma in the proof of Theorem 4.3.

###### Lemma 4.4.

Let be a Lévy process with characteristic exponent , Blumenthal–Getoor index , and satisfying for some . We assume moreover that is asymptotically dominated by , i.e., there exists a constant such that for all . Then, for all , there exists a constant such that

 E[|t−1/βLt|q]≤Mq,∀0

Upper bounds such as (18) are known as moment estimates and play a crucial role when studying the local smoothness of the sample paths of Lévy processes [3, 25, 11]. Moment estimates and extensions thereof have been studied by several authors [48, 18, 45]. Lemma 4.4 can be deduced from known estimates. In particular, it is a consequence of [46, Theorem 2.9], which considers the more general class of Lévy-type processes. For completeness, a self-contained proof in the case of Lévy processes is provided in Appendix B. Note that the hypothesis that is asymptotically dominated by is automatically satisfied for LSS processes (by Proposition 4.2) but is not true for arbitrary Lévy processes.

###### Proof of Theorem 4.3.

Fix a sequence of positive reals such that and define for . We will prove that which implies that as since the sequence is arbitrary. Equation (17) then follows since for .

Due to the sector condition (8), we have that

where we defined . Hence for all and ,

 |ΦXn(ξ)| (19)

where the third equality uses that for .

By Proposition 4.2, the characteristic exponent of satisfies as . In particular, there exists such that for . Let us now define the function by for and for . For large enough such that we have, for all ,

 (20)

Indeed, (20) is obvious for since for . For , we also have (since ), hence and (20) readily follows by definition of .

By assumption on , converges in distribution to which implies the pointwise convergence for all as . Equation (4.2) together with (20) implies that is uniformly dominated by the integrable function for large enough. Therefore, in -norm by Lebesgue’s dominated convergence theorem.

Denote by the pdf of and by the pdf of for (those are well-defined since and are integrable). Writing and as the inverse Fourier transforms of and and using the triangle inequality, we obtain for all and ,

 2π∣∣pn(x)−qβ(x)∣∣=∣∣∣∫R(ΦXn(ξ)e−ixξ−ΦYβ(ξ)e−ixξ)dξ∣∣∣≤∫R∣∣ΦXn(ξ)−ΦYβ(ξ)∣∣dξ.

Since in -norm, this implies the pointwise—in fact, uniform—convergence for all as (this fact is well-known, see for instance [75, Corollary 1.2.4]). By Scheffé’s lemma, this in turns implies the convergence in -norm.

Writing as the inverse Fourier transform of as above, we get for large enough and similarly for . Finally, according to Lemma 4.4, for , there exists such that for large enough such that . We then conclude using [33, Theorem 1] that . ∎

###### Example.

If is an SS process, it is obviously LSS and Theorem 4.3 then implies that

 h(Lαt)=1αlogt+h(Lα1)+ot(1),

where when . This specific case can be proved directly using the stability property as was done by Ghourchian et al. in [32, Proposition 1]. Theorem 4.3 thus significantly generalizes this result to all LSS processes.

###### Remark.

The key step in the proof of Theorem 4.3 is to prove the convergence of the sequence to not only in law but in the (stronger) sense that the sequence of probability density functions converges uniformly to the density of . This is closely related to the literature on local limit theorems, whose goal is to characterize the convergence of normalized sums of i.i.d. random variables in terms of the convergence of the pdfs, and includes notably uniform convergence [34], convergence in total variation [56], and convergence in relative entropy (also known as Kullback–Leibler divergence) which was investigated for Gaussian limits [5] and non-Gaussian stable limits [10]. See also [9] for a recent unifying perspective on local limit theorems.

Let be a Lévy process with increments at integral times denoted by for . Then, we have that is a sum of i.i.d. random variables and local limit theorems readily apply to the study of the asymptotic behavior of for . In contrast, we characterize here the asymptotic behavior of the differential entropy of at small times

and our work can thus be seen as the local counterpart of classic entropic central limit theorems

[5, 10]. Further discussion about the local versus asymptotic properties of Lévy processes and their generalizations can be found in [22, Chapter 7].

### 4.3 An Upper Bound on the Small-Time Entropy of Lévy Processes

The specific case of LSS Lévy processes studied in the previous section excludes both Lévy processes with which are not LSS and Lévy processes with . In this section, we consider a general Lévy process and upper bound the decay of at small times. Specifically, the following theorem gives an upper bound on the limit superior of when in terms of the Blumenthal–Getoor index . This is achieved by “approximating” with an LSS process and applying Theorem 4.3.

###### Theorem 4.5.

Let be a Lévy process with Blumenthal–Getoor index and such that has a finite absolute moment for some . Then,

 limsupt→0+h(Lt)log(1/t)≤−1β, (21)

with the convention . In particular, for , we have that

 limt→0+h(Lt)log(1/t)=−∞. (22)
###### Proof.

The case is covered by Proposition 3.7 (ii) so we henceforth assume that . For , let be a SS process with parameter , independent of , and define . We denote by , , and the characteristic exponents of , , and respectively. By definition, there exists such that . Moreover, since and by definition of , we have that when . This implies that when , hence is LSS by Proposition 4.2.

For two independent absolutely continuous random variables such that has finite differential entropy, we have that222The finiteness of implies that is well-defined in . This is therefore the case for in the proof. Note that the theorem is trivially true when . . Hence

 h(Lt)≤h(Lt+Lαt)=h(˜Lt)=−1αlog(1/t)+Ot(1), (23)

where the last equality is by Theorem 4.3 applied to (the assumptions of Theorem 4.3 are easily satisfied from the assumptions on ). Dividing by for and taking the limit superior on both sides of (23) yields

 limsupt→0+h(Lt)log(1/t)≤limsupt→0+{−1α+ot(1)}=−1α,

which concludes the proof after letting . ∎

## 5 Examples and Applications

We apply our previous findings to specific subfamilies of Lévy processes. We consider non-stable Lévy processes with positive Blumenthal-Getoor indices in Section 5.1 and gamma processes, for which , in Section 5.2.

### 5.1 Layered and Tempered Stable Processes

Layered stable processes and tempered stable processes were respectively introduced by Christian Houdré and Reiichiro Kawai [38] and by Jan Rosinski [60]. In both cases, we characterize the asymptotic behavior of the entropy, thus demonstrating the applicability of Theorem 4.3 to Lévy processes that were not covered by previous results. For simplicity, we restrict ourselves to symmetric Lévy processes.

#### Layered Stable Processes

###### Definition 5.1.

A symmetric Lévy process is said to be a layered stable process with indices if it has no Gaussian part ( in (5)) and if its Lévy measure can be written as where is an even and continuous such that

 q(t)∼t→0a0|t|α0+1andq(t)∼|t|→∞a∞|t|α∞+1 (24)

for come constants .

Definition 5.1 is adapted from [38, Definition 2.1] to the ambient dimension . Stable processes correspond to the specific case where for some constant and . As was studied in details in [38], the index governs the short-term behavior of the Lévy process, while captures long-terms structures. One should therefore not be surprised to see in the next proposition that the entropy of a layered stable process is asymptotically determined by its index .

###### Proposition 5.2.

Let be a layered stable process with parameters . Then, is LSS with Blumenthal–Getoor index and

 h(Lt)=1α0logt+h+ot(1) (25)

for some constant , where vanishes for .

###### Proof.

From (5), we easily deduce that the characteristic exponent of a symmetric Lévy process without Gaussian part is given by

 Ψ(ξ)=∫R(cos(tξ)−1)ν(dt),∀ξ∈R. (26)

We decompose with

 Ψ0(ξ)=∫|t|≤1(cos(tξ)−1)ν(dt)andΨ∞(ξ)=∫|t|>1(cos(tξ)−1)ν(dt).

Then, after the change of variable for , we get

 Ψ0(ξ)=−(∫R(1−cosu)q(u/ξ)|ξ|α0+11|u|≤|ξ|du)|ξ|α0:=−(∫Rgξ(u)du)|ξ|α0.

Then, due to (24) for , we have that for all when . Moreover, again according to (24) and using the continuity of , there exists such that for all . Hence, we deduce that

 0≤gξ(u)≤1|u|≤|ξ|M(1−cosu)|u|α0+1≤M(1−cosu)|u|α0+1,

the latter being integrable on . Lebesgue’s dominated convergence theorem then ensures that when , and finally,

 Ψ0(ξ)∼|ξ|→∞−γ|ξ|α0. (27)

Moreover, (24) and the continuity of implies the existence of such that for all . This implies that

 |Ψ∞(ξ)|=∫|t|>1(1−cos(ξt))q(t)dt≤2M∫|t|>1dt|t|α∞+1, (28)

which is a finite constant independent of .

Combining (27) and (28), we deduce that as which implies by Proposition 4.2 that is LSS with Blumenthal–Getoor index .

Finally, for all by [38, Proposition 2.1]. Hence satisfies all the assumptions of Theorem 4.3 and we conclude that (25) holds. ∎

#### Proper Tempered Stable Processes

###### Definition 5.3.

A symmetric Lévy process is said to be a proper tempered stable process with parameters and if it has no Gaussian part and if its Lévy measure can be written as

 ν(dt)=q(|t|p)dt|t|α+1 (29)

for some completely monotone function333See [65] for a general reference on completely monotone functions. such that

 ∫10x1−αq(x)dx<∞,∫∞1x−1−αq(x)dx<∞,q(x)⟶x→∞0,  and  q(x)⟶x→0+c>0. (30)

For this definition, we follow the monograph [35], in particular Definitions 3.1 and 3.2 specialized to the ambient dimension . Stable processes correspond to the case where is identically equal to a positive constant, although this is excluded by the condition that vanishes at infinity. The effect of is to temper the asymptotic behavior of , which impacts the heavy-tailedness of the Lévy process. We refer the interested reader to [35, Chapter 1] for practical motivations and for historical references on tempered stable processes.

###### Proposition 5.4.

Let be a proper tempered stable process with parameters and . Then, is LSS with Blumenthal–Getoor index and

 h(Lt)=1αlogt+h+ot(1) (31)

for some constant , where vanishes for .

###### Proof.

The proof of Proposition 5.4 is similar to the one of Proposition 5.2, therefore we only present the main arguments. The Blumenthal–Getoor index of is characterized in [35, Lemmata 3.22 and 3.23]. As we did for layered stable processes, one can use the behavior of the Lévy measure at the origin and at infinity to deduce that the Lévy exponent, given by (26), satisfies when for some . We therefore deduce from Proposition 4.2 that is LSS of order . Finiteness of an absolute moment , is shown for all in [35, Theorem 3.15]. Thus, satisfies the assumptions of Theorem 4.3 from which we obtain (31). ∎

### 5.2 Gamma Processes

In the case where , Theorem 4.5 guarantees that diverges to super-logarithmically. The decay rate can in fact be significantly faster as illustrated in this section by the gamma process, for which we show that it is asymptotically equivalent to for some constant .

The gamma distribution with scale parameter

and shape parameter has characteristic function given by and is thus infinitely divisible. It defines a Lévy process , called a gamma process, whose characteristic exponent satisfies as . Hence its Blumenthal–Getoor index is zero.

###### Proposition 5.5.

Let be a gamma process with scale parameter and shape parameter . Then we have as ,

 h(Lt)=−1τt−logt+logθτ−γ+1+ot(1). (32)
###### Proof.

For all , is gamma-distributed with scale parameter and shape parameter , hence its differential entropy is