DeepAI

Concentration bounds for empirical conditional value-at-risk: The unbounded case

In several real-world applications involving decision making under uncertainty, the traditional expected value objective may not be suitable, as it may be necessary to control losses in the case of a rare but extreme event. Conditional Value-at-Risk (CVaR) is a popular risk measure for modeling the aforementioned objective. We consider the problem of estimating CVaR from i.i.d. samples of an unbounded random variable, which is either sub-Gaussian or sub-exponential. We derive a novel one-sided concentration bound for a natural sample-based CVaR estimator in this setting. Our bound relies on a concentration result for a quantile-based estimator for Value-at-Risk (VaR), which may be of independent interest.

• 3 publications
• 21 publications
• 4 publications
• 22 publications
01/04/2019

Risk-aware Multi-armed Bandits Using Conditional Value-at-Risk

Traditional multi-armed bandit problems are geared towards finding the a...
02/27/2019

Improved Concentration Bounds for Conditional Value-at-Risk and Cumulative Prospect Theory using Wasserstein distance

Known finite-sample concentration bounds for the Wasserstein distance be...
06/26/2020

PAC-Bayesian Bound for the Conditional Value at Risk

Conditional Value at Risk (CVaR) is a family of "coherent risk measures"...
12/22/2019

Estimation of Spectral Risk Measures

We consider the problem of estimating a spectral risk measure (SRM) from...
07/01/2020

The Restricted Isometry of ReLU Networks: Generalization through Norm Concentration

While regression tasks aim at interpolating a relation on the entire inp...
03/23/2022

Towards Scalable Risk Analysis for Stochastic Systems Using Extreme Value Theory

We aim to analyze the behaviour of a finite-time stochastic system, whos...
04/28/2020

Avoiding zero probability events when computing Value at Risk contributions: a Malliavin calculus approach

This paper is concerned with the process of risk allocation for a generi...

1 Introduction

In several practical decision problems, the presence of uncertainty complicates the decision making process as decisions typically are required to be taken before the uncertainty is resolved. Traditionally, this difficulty is overcome by averaging the costs (or rewards) over all possible realizations of the uncertainty, and then optimizing the averaged cost thus obtained. However, it has been argued that considering averaged outcomes is not appropriate in situations where low-probability events such as financial crashes and category 4 hurricanes can cause huge costs. The possibility of occurrence of such tail events has led to the introduction of risk measures such as Value-at-Risk (VaR) and Conditional Value-at-Risk (CVaR) for quantification of risk. In financial risk management, the VaR of a risky portfolio at a confidence level

is a loss threshold such that the probability of the loss exceeding the threshold is no greater than . The CVaR of a portfolio at a confidence level is the expected loss on the portfolio conditioned on the event that the loss exceeds the VaR. Loosely speaking, VaR quantifies the maximum loss that can occur in the absence of a catastrophic tail event, while the CVaR gives the expected loss given the occurrence of such a tail event. CVaR has several desirable properties as a risk measure. In particular, it is a convex, coherent risk measure (see the survey paper [4] and references therein). As a result, CVaR continues to receive increasing attention in operations research, mathematical finance, and decision science for problems involving risk quantification or risk minimization.

In most applications involving uncertainty, the distributions characterizing the underlying uncertain factors are not known, and risk measures such as CVaR have to be estimated from sampled values of the random variable of interest. This is true, for instance, in a multi-armed bandit problem [5, 3] in which pulling an arm leads to a random loss, and one seeks to identify the arm whose loss random variable has the lowest CVaR by observing a sample of outcomes that result from multiple arm pulls. An obvious estimator for the CVaR of a distribution is the sample CVaR of an i.i.d. sample drawn from the distribution. Naturally, one seeks error bounds for the estimator that help to understand the trade-off between accuracy and sample size. Previous results on CVaR estimation either provide asymptotic error bounds for a general r.v. [7], or provide non-asymptotic error bounds that hold with high probability, but under the stringent assumption that the underlying r.v. is bounded [2, 8].

In this paper, we consider the problem of estimating the CVaR of an unbounded, albeit sub-Gaussian or sub-exponential random variable. Sub-Gaussian r.v.s include bounded, Gaussian and any other r.v. whose tail decays as fast as a Gaussian. On the other hand, sub-exponential r.v.s include exponential, Poisson and squared-Gaussian r.v. and are characterized by a tail heavier than Gaussian, resembling that of an exponential distribution. To the best of our knowledge, there are no concentration bounds for CVaR estimator for these two popular classes of unbounded distributions. We believe, imposing a tail decay assumption (sub-Gaussian or sub-exponential) is not restrictive, as such an imposition is common in concentration results for sample mean. Also, the task of CVaR estimation is more challenging in comparison, as it relates to a tail event. We derive a one-sided concentration bound for the empirical CVaR of an i.i.d. sample. Our bound relies on one of the two concentration results (of possibly independent interest) that we provide for a quantile-based estimator for VaR.

The rest of the paper is organized as follows: Section 2 introduces VaR, CVaR and their estimators from i.i.d. samples, Section 3 presents the concentration bounds for VaR and CVaR estimators. Section 4 provides detailed proofs of the concentration results, and Section 5 concludes the paper.

2 Background

Given a r.v.

with cumulative distribution function (CDF)

, the VaR and CVaR  111For notational brevity, we omit from and whenever the r.v. can be understood from the context. at level are defined as follows:

 vα(X) =inf{ξ:P[X≤ξ]≥α} % and (1) cα(X) =vα(X)+11−αE[X−vα(X)]+, (2)

where we have used the notation that for a real number Typical values of chosen in practice are and . Note that, if has a continuous and strictly increasing CDF, then is a solution to the following i.e., CVaR also admits another form under the following assumption:

(A1) The r.v. is continuous and has strictly increasing CDF.

If (A1) holds and has a positive density at then admits the following equivalent form (cf. [7]):

 cα(X)=E[X|X≥vα(X)].

Let denote i.i.d. samples from the distribution of . Then, the estimates of VaR and CVaR at level , denoted by and , are formed as follows [6]:

 ^vn,α =^F−1n(α):=inf{x:^Fn(x)≥α} (3) ^cn,α =^vn,α+1n(1−α)n∑i=1(Xi−^vn,α)+, (4)

where is the empirical distribution function of . Note that, from the order statistics , the empirical VaR can be computed as follows:

3 Concentration bounds

In this section, we present four concentration bounds. The first two bounds are for the VaR estimator given in (3) and these bounds do not impose any restrictions on the underlying distribution. The next two concentration results are for the CVaR estimator given in (4), and for these results, we assume that the underlying distribution is either sub-Gaussian or sub-exponential (see Definitions 33 below).

In each of the result presented below, the estimates are calculated using i.i.d. samples drawn from the r.v. with CDF and for a given (VaR concentration bound) Let and Define and . Further, let and , where is defined by (3). Then,

 P[vα(X)∈[an,bn]]≥(1−2exp(−n1−2s8)).

Note that, the above concentration bound is free of any distribution dependent parameters.

Proof.

See Section 4.1. ∎

The following result presents another concentration bound for the VaR estimator which will have distribution parameters in the bound. However, unlike Proposition 3, the result presented below is symmetric, and more importantly, bounds the estimation error directly. [VaR concentration bound] Suppose that (A1) holds. For any we have

where is a constant that depends on the value of the density of the r.v. in a neighbourhood of VaR.

Proof.

See Section 4.2. ∎

The bound above implies that to estimate the VaR to an accuracy of , one would require an order number of samples. Notice that no restrictive assumptions on the tail of the underlying distribution are made in arriving at the concentration bounds for VaR in Propositions 3 and 3. However, for establishing concentration bounds for the CVaR, which involves conditioning on a tail event, it is necessary to assume that the distribution is not heavy-tailed. In fact, even for the case of estimating the expected value of a r.v., exponential concentration bounds are available under an assumption that restricts the tail to be light (cf. Chapter 2 of [1]).

In this paper, we present concentration bounds under two popular assumptions on the tail of a r.v. The first restricts the r.v. to be sub-Gaussian, while the second requires the same to be sub-exponential. These two classes of r.v.s include bounded r.v.s and more importantly, several unbounded r.v.s as well. Sub-Gaussian r.v.s include the Gaussian r.v.s as well as several other r.v.s whose moment generating functions do not exceed that of a Gaussian, while sub-exponential r.v.s include heavier tailed r.v.s. These two notions are made precise in the following definitions.

A r.v. with is said to be -sub-Gaussian if

 E[exp(λX)]≤exp(λμ+λ2σ22),∀λ∈R.

A r.v. with mean is said to be -sub-exponential if

 E[exp(λX)]≤exp(λμ+λ2σ22),∀|λ|<1b.

It is worth noting that all sub-Gaussian r.v.s are sub-exponential, but the converse is not true. The following result presents a one-sided concentration bound for the CVaR estimator in (4), for the case when the underlying r.v. is sub-Gaussian. [CVaR concentration bound: sub-Gaussian case] Suppose that (A1) holds. Let and be a -sub-Gaussian r.v. with mean . Suppose that is large enough to ensure and the sub-Gaussian parameter satisfies Then, for any we have

 P[^cn,α−cα>ϵ]≤exp(−nϵ(1−α)(vα−μ)2σ2) +2exp(−2nc1ϵ2)+2exp(−2nc2ϵ) +exp(−2nϵ(1−α)2), (5)

where and are constants that depend on the value of the density of the r.v. in a neighbourhood of VaR.

Proof.

See Section 4.3. ∎

Suppose that the accuracy is greater than . Then, it is apparent that the dominant terms on the RHS of (5) are those involving an exponential with . Further, with probability (w.p.) at least , when the number of samples is of the order . On other hand, an order number of samples are enough to ensure that w.p. at least . Hence, CVaR estimation requires more samples in comparison to VaR, when . In the complementary case, i.e., when , both VaR and CVaR estimate can be -accurate w.p. , if the number of samples is of the order .

Next, we analyse the concentration of the CVaR estimator in (4) for the case when the underlying r.v. is sub-exponential. [CVaR concentration bound: sub-exponential case] Suppose that (A1) holds. Let be a -sub-exponential r.v. with mean . Suppose that is large enough to ensure and the parameter satisfies where where Then for any we have

 P[^cn,α−cα>ϵ]≤exp(−nϵ(1−α)mb2)+2exp(−2nc1ϵ2)+2exp(−2nc2ϵ)+exp(−2nϵ(1−α)2), (6)

where and are as in Proposition 3. From the result above, it is apparent that the rate of CVaR concentration for sub-exponential r.v.s matches that of sub-Gaussian ones.

Proof.

See Section 4.4. ∎

4 Proofs

In this section, we present the proofs of the results presented in Section 3.

4.1 Proof of Proposition 3

Proof.

Recall the Dvoretzky-Kiefer-Wolfowitz (DKW) inequality, which provides a finite-sample bound on the distance between the empirical distribution and the true distribution: For any ,

 P[supx∈R|^Fn(x)−F(x)|>ϵ]≤2e−2nϵ2.

Consider the following event

 A={supx∈[an,bn]∣∣^Fn(x)−F(x)∣∣≤14ns},

with and as defined in the theorem statement. By the DKW inequality, we have

 P[A]≥1−2exp(−2n16n2s)=1−2exp(−n1−2s8). (7)

On the event , we have

 F(an) (a)=lima↑anF(a)(b)≤lima↑an(^Fn(a)+14ns)(c)≤α−+14ns=α−14ns<α, and F(bn) ≥α+−14ns=α+14ns>α,

where (a) follows from continuity of , (b) follows from the definition of the event , and (c) follows from the definition of . Thus, and the main claim follows from the lower bound on in (7). ∎

4.2 Proof of Proposition 3

Proof.
 ≤P[^Fn(vα+ϵ)≤α]+P[^Fn(vα−ϵ)≥α] =P[F(vα+ϵ)−^Fn(vα+ϵ)≥F(vα+ϵ)−α]+P[^Fn(vα−ϵ)−F(vα−ϵ)≥α−F(vα−ϵ)] (a)≤exp(−2n(F(vα+ϵ)−F(vα))2)+exp(−2n(F(vα)−F(vα−ϵ))2) ≤2exp(−2nδ2ϵ),

where is due to the DKW inequality, and
Given that the density exists, we have

 F(vα+η1)−F(vα−η2)=f(¯v)(η1+η2),

for some . Using the identity above for the two expressions inside , we obtain

 δϵ=min(f(¯v1),f(¯v2))×ϵ.

for some and The claim follows. ∎

4.3 Proof of Proposition 3

We first prove a more general result without restricting the sub-Gaussian parameter and the main claim in Proposition 3 follows in a straightforward fashion.

[General CVaR concentration bound: sub-Gaussian case] Assume (A1). Let be a -sub-Gaussian with mean . Suppose that . Then, for any , we have

 P[^cn,α−cα>ϵ] ≤exp(−nϵ(1−α)(vα−μ)2σ2)[α+exp(−(vα−μ)22σ2)]n+2exp(−2nδ2ϵ1) +2exp(−2nδ2ϵ2)+exp(−2nϵ(1−α)2), (8)

where and

Proof.

First, we bound the estimate . Notice that

 ^cn,α =^vn,α+1n(1−α)n∑i=1(Xi−^vn,α)+ =vα+1n(1−α)n∑i=1(Xi−vα)++(^vn,α−vα)+1n(1−α)n∑i=1[(Xi−^vn,α)+−(Xi−vα)+] (9)

The last term on the RHS of (9) can be re-written as follows:

 1n(1−α)n∑i=1[(Xi−^vn,α)+−(Xi−vα)+] =1n(1−α)n∑i=1[(vα−^vn,α)I{Xi≥^vn,α}] +1n(1−α)n∑i=1(Xi−vα)[I{Xi≥^vn,α}−I{Xi≥vα}], (10)

and

 1n(1−α)n∑i=1[(vα−^vn,α)I{Xi≥^vn,α}] =^vn,α−vα1−α[[^Fn(^vn,α)−1]−1nn∑i=1I{Xi=^vn,α}] =^vn,α−vα1−α[^Fn(^vn,α)−1] w.p. 1. (11)

The last equality above uses the fact that takes values zero w.p. , since is continuous, for each .

Combining (9), (10) and (11), we obtain

 ^cn,α =vα+1nn∑i=1(Xi−vα)+1−α+(^vn,α−vα)+^vn,α−v1−α[^Fn(^vn,α)−1] =vα+1n(1−α)n∑i=1(Xi−vα)++Bn, (12)

where

 BnΔ=^vn,α−vα1−α[^Fn(^vn,α)−α]+1nn∑i=1Xi−vα1−α[I{Xi≥^vn,α}−I{Xi≥vα}].

From (12), we have

 ^cn,α−cα =[vα+1nn∑i=1(Xi−vα)+1−α−cα]+Bn =11−α[1nn∑i=1(Xi−vα)I{Xi≥vα}−E[(X−vα)I{X≥vα}]]+Bn. (13)

For notational convenience, let

 Yi =(Xi−vα)I{Xi≥vα}, Y =(X−vα)I{X≥vα} % and An =11−α(1nn∑i=1Yi−E[Y]).

It is easy to see that ’s are i.i.d., non-negative, and , . We now proceed to bound , using (13), as follows:

 P[^cn,α−cα>ϵ] =P[An+Bn>ϵ]≤P[An>ϵ/2]+P[Bn>ϵ/2]. (14)

For handling , we bound the moment generating function of r.v. as follows:

 E[eλYi/n]=E[eλn(Xi−vα)I{Xi≥vα}] =∫vα−∞fX(x)dx+∫∞vαeλn(x−vα)fX(x)dx ≤FX(vα)+e−λnvα∫∞−∞eλnxfX(x)dx (a)≤α+e−λnvα+λnμ+λ2σ22n2, (15)

where is due to the sub-Gaussianity of . Thus,

 P[An>ϵ2]=P[1nn∑i=1Yi>(1−α)ϵ2+E[Y]] (b)≤Πni=1E[eλYi/n]eλ((1−α)ϵ/2+E[Y])(c)≤[α+e−λnvα+λnμ+λ2σ22n2]neλ((1−α)ϵ/2+E[Y]), (16)

where uses Markov’s inequality and follows from (15). Notice that (16) holds for any . However, for the bound on the RHS above to be meaningful, we require that . Now, maximizing over , we obtain . Substituting the value of in (16), we obtain

 P[An>ϵ/2]≤exp(−n(vα−μ)σ2[(1−α)ϵ2+E[Y]])[α+exp(−(vα−μ)22σ2)]n. (17)

For handling the term in (14), we bound as follows. Using the inequality above, we bound as follows:

 |Bn| ≤|vα−^vn,α|1−α|α−^Fn(^vn,α)|+|vα−^vn,α|1−α|^Fn(vα)−^Fn(^vn,α)| =|vα−^vn,α|1−α[|F(vα)−^Fn(^vn,α)|+|^Fn(vα)−F(vα)−^Fn(^vn,α)+F(vα)|] ≤|vα−^vn,α|1−α[2|^Fn(^vn,α)−F(vα)|+|^Fn(vα)−F(vα)|]. (18)

The first inequality above uses the following fact:

 ∣∣1nn∑i=1Xi−vα1−α[I{Xi≥^vn,α}−I{Xi≥vα}]∣∣≤11−α|vα−^vn,α||^Fn(vα)−^Fn(^vn,α)|.

Using (18), we have

 P[Bn>ϵ2] ≤P[|Bn|>ϵ2]≤P[|^vn,α−vα|1−α(2|^Fn,α(^vn,α)−F(vα)|+|^Fn(vα)−F(vα)|)>ϵ2]

It is easy to see that Hence,

 P[Bn>ϵ2] ≤P[|Bn|>ϵ2]≤P[|^vn,α−vα|1−α(2n+|^Fn(vα)−F(vα)|)>ϵ2] ≤P[2n1(1−α)|^vn,α−vα|>ϵ4]+P[11−α|^vn,α−vα||^Fn(vα)−F(vα)|>ϵ4].

Let . Then, we have

 P[|^vn,α−vα||^Fn(vα)−F(vα)|1−α>ϵ4] ≤P[|^vn,α−vα||^Fn(vα)−F(vα)|1−α>ϵ4,Dn]+P[Dcn] ≤P[|^Fn(vα)−F(vα)|>√ϵ(1−α)]+P[Dcn] (d)≤exp(−2nϵ(1−α)2)+P[Dcn],

where is due to the DKW inequality. Therefore,

 P[Bn>ϵ/2]≤P[|^vn,α−vα|>n(1−α)ϵ/8]+P[|^vn,α−vα|>√ϵ/4]+exp(−2nϵ(1−α)2). (19)

Using (14), (17) and (19), we obtain

 P[^cn,α−cα>ϵ] ≤exp(−n(vα−μ)σ2[(1−α)ϵ2+E[Y]])[α+exp(−(vα−μ)22σ2)]n +P[|^vn,α−vα|>n(1−α)ϵ8]+P[|^vn,α−vα|≥√ϵ4]+exp(−2nϵ(1−α)2) (a)≤exp(−nϵ(1−α)(vα−μ)2σ2)[α+exp(−(vα−μ)22σ2)]n+2exp(−2nδ2ϵ1)) +2exp(−2nδ2ϵ2))+exp(−2nϵ(1−α)2),

where are as defined in the statement of the proposition, and is due to Proposition 3 and the fact that . ∎

Proof of Proposition 3

Proof.

For we note that Given that the density exists, we have

 F(vα+η1)−F(vα−η2)=f(¯v)(η1+η2),

for some . Using the identity above for the two expressions inside , we obtain

 δϵ1=min(f(¯v1),f(¯v2))×n(1−α)ϵ8.

for some and . Along similar lines, it is easy to infer that

 δϵ2=min(f(¯v3),f(¯v4))×√ϵ4,

for some and . The claim follows. ∎

4.4 Proof of Proposition 3

Proof.

Observe that, in the proof of Proposition 4.3, sub-Gaussianity is used is bounding following (14) there. Here, we bound the same using sub-exponential assumption.

Recall that where and Starting as in the derivation of (15),

 E[eλYi]=E[eλ(Xi−vα)I{Xi≥vα}] ≤FX(v