Robustness of Dynamical Quantities of Interest via Goal-Oriented Information Theory

06/21/2019 ∙ by Jeremiah Birrell, et al. ∙ University of Massachusetts Amherst 0

Variational-principle-based methods that relate expectations of a quantity of interest to information-theoretic divergences have proven to be effective tools for obtaining robustness bounds under both parametric and non-parametric model-form uncertainty. Here, we extend these ideas to utilize information divergences that are specifically targeted at the quantity-of-interest being investigated; leveraging this specificity results in tighter robustness bounds and expands the class of problems that can be treated. We find the method especially useful for problems involving unbounded time-horizons, a case where established information-theoretic methods only produce trivial bounds. General problem types that can be treated within this framework include the expected value and distribution of a stopping time, time averages, and exponentially discounted observables. We illustrate these results with several examples, including option pricing, stochastic control, semi-Markov processes, and expectations and distributions of hitting times.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Variational-principle-based methods, relating expectations of a quantity of interest to information-theoretic divergences, have proven to be effective tools for obtaining robustness bounds, i.e. for uncertainty quantification (UQ), in the presence of both parametric and non-parametric model-form uncertainty chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . In the present work, we build upon these ideas by utilizing information-theoretic quantities, the so-called goal-oriented relative entropy and Fisher information, that are targeted at a specific quantity-of-interest (QoI) under investigation. Tailoring the method to the QoI results in tighter UQ bounds, and allows for the derivation of UQ bounds in cases where previous methods fail to apply.

In this paper, UQ will refer to the following general problem:

One has a baseline model, described by a probability measure

, and a (family of) alternative model(s), . The base model is typically the ‘simpler’ model, meaning one can calculate QoI’s exactly, or it is relatively inexpensive to simulate, but it may contain many sources of error and uncertainty; it may depend on parameters with uncertain values (obtained from experiment, Monte-Carlo simulation, variational inference, etc.) or is obtained via some approximation procedure (dimension reduction, neglecting memory terms, linearization, asymptotic approximation, etc.). Any quantity of interest computed from therefore has significant uncertainty associated with it. In contrast, the family of alternative models is generally thought of as containing the ‘true’ (or at least, more precise) model, but due to its complexity or a lack of knowledge, it is considered intractable. Our UQ goal is then to, for a given QoI (real-valued function), , bound the error incurred from using in place of , i.e.

(1)

Here, and in the following, will denote the expectation under the probability measure . We will also refer to (resp. ) as the base model (resp. alternative model) QoI.

As examples, could be the failure-time of a system (or a more general stopping time), the value of some asset, or a cost-functional; a bound on Eq. (1) is then a performance guarantee, given the modeling uncertainty expressed by the family . See Section 5 for concrete examples.

The primary philosophy behind the methods derived below is that the bound Eq. (1) should be specialized to the QoI, , and the family

as much as possible, while still involving quantities that can be effectively computed/estimated. In short, one does not necessarily care about

all differences between, and , but only the differences insofar as they impact the expectation of the QoI.

1.1 Summary of Results

Our new results all rest upon an improvement to the Gibbs-Variational-Principle-based UQ bounds that were developed and employed in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B ; this improvement is the content of Section 3. Specifically, Theorem 3.1 bounds (under suitable assumptions) the difference in expectation of a QoI, , with respect to a base model, , and an alternative model, , as follows:

(2)

Here is the QoI centered under , , is the cumulant generating function of under , denotes the distribution of on under , and similarly for , and

denotes the relative entropy (i.e. Kullback-Leibler divergence). See Section

2 for detailed definitions and further background.

Again, is typically taken to be a tractable model, so that the cumulant generating function can be computed or bounded explicitly. As for the second term, is generally unobtainable, as it requires detailed knowledge of the distribution of under the intractable alternative model, . However, Eq. (2) becomes a practical tool for UQ when it is combined with the data processing inequality (again, see Theorem 3.1). Specifically, if is -measurable, where is a sigma sub-algebra of the full sigma algebra of the probability space, then

(3)

Here, denotes the restriction of the probability measure to the sigma sub-algebra. For example, many of our applications will involve some -stopping time, , in which case we take , the sigma-algebra up to the stopping time.

Eq. (3) gives one a great deal of freedom when weakening the robustness bound Eq. (2); ideally, this freedom can be exploited to obtain bounds that depend on an explicitly computable relative entropy, while still maintaining as much information regarding as possible (i.e. enlarging as little as possible). With this aim in mind, we call a goal-oriented relative entropy, to contrast it with the non-goal-oriented quantity . The practical benefit of using goal-oriented quantities will be demonstrated throughout our examples; for a particularly simple and clear demonstration, see Section 5.1.1, especially Figure 1.

Clearly, it is only useful to weaken the robustness bound Eq. (2) via the data processing inequality when is computable in some sense. In practice, we will require a bound of the following form to hold: for some -measurable . Given this, we obtain UQ bounds that only involve expectations under the (tractable) base model, (see Theorem 3.2):

(4)

In fact, (under appropriate assumptions) taking makes Eq. (4) an equality. Of course, this choice is often impractical to work with; one should aim to select appropriate and so that the right-hand-side of Eq. (4) is comparatively simple to compute/estimate, while at the same time keeping the inequalities as tight as possible. Apart from Eq. (4), other techniques for eliminating the -dependence of the bounds are also useful; see Remark 4 and the example in Section 5.5.2.

Eq. (2) and Eq. (4) are non-perturbative UQ-bounds. For parametric families, one can similarly obtain perturbative UQ-bounds (i.e. sensitivity bounds) in terms of so-called goal-oriented Fisher information; see Appendix A.

Our primary application of these results will be to QoI’s up to a stopping time. In this setting, one has a time-dependent process, , adapted to a filtration , and a -stopping time , and the goal is to bound the expectation of the stopped process, . This general framework is discussed in Section 3.1.

In Section 4 we discuss in detail several classes of problems that fit within the framework of Section 3.1. These are listed in Table 1 and summarized below:

QoI
1
2
3
4
5
Table 1:
  1. Bounding the relative entropy up to a stopping time.

  2. Bounding the distribution of a stopping time.

  3. Bounding the expected value of a stopping time.

  4. Bounding the expectation of exponentially discounted observables.

  5. Bounding the expectation of time averages. (These were treated previously in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . We show that our framework leads to the same bounds in this case.)

For example, in Corollary 6 we obtain the following bounds on the expectation of a stopping time, :

(5)

for any alternative model, , satisfying a relative entropy bound of the form

(6)

for some and all . As we will show, the bound Eq. (5) does not require a-priori knowledge that is finite.

Appendix B contains formulas for the relative entropy up to a stopping time for Markov processes in both discrete and continuous time. In particular, we will see that a bound of the form Eq. (6) is not uncommon when the base and alternative models come from Markov processes.

Finally, we illustrate the use of these results in Section 5 through several examples. Specifically, we study:

  1. The distribution and expectation of hitting times of Brownian motion with constant drift, as compared to Brownian motion perturbed by a non-constant drift.

  2. Expected hitting times of a Ornstein-Uhlenbeck process perturbed by a non-constant drift.

  3. Exponentially-discounted cost for stochastic control systems.

  4. Semi-Markov perturbations of a -queue.

  5. The value of American put options in a variable-interest-rate environment.

2 Background on Uncertainty Quantification via Variational Principles

Here we record important background information on the variational-principle approach to UQ; the new methods presented in this paper will be built upon these foundations, starting in Section 3.

First, we fix some notation. Let be a probability measure on a measurable space .

will denote the extended reals and we will refer to a random variable

as a quantity-of-interest (QoI). The cumulant generating function of is defined by

(7)

where denotes the natural logarithm. We use the continuous extensions of to and of to .

The following notation will be used for the set of real-valued QoIs with a well-defined and finite moment generating function (MGF) under

:

(8)

It is not difficult to prove (see e.g. DemboZeitouni ) that for , the cumulant generating function is a convex function, finite and infinitely differentiable in some interval with , equal to outside of , and such have moments of all orders.

If we write

(9)

for the centered quantity of mean . We will often use the cumulant generating function for a centered observable :

(10)

Recall that .

Recall also that the relative entropy (or Kullback-Leibler divergence) of another probability measure, , with respect to is defined by

(11)

It has the property of a divergence, that is and if and only if ; see, for example, dupuis2011weak for this and further properties.

The starting point of our approach is a non-goal-oriented UQ bound, derived in chowdhary_dupuis_2013 ; DKPP (a similar inequality is used in the context of concentration inequalities, see e.g. BLM , and was also used independently in BREUER20131552 ; GlassermanXu ). We summarize the proof here for completeness.

Proposition 1 (Gibbs information inequality)

Let , . Then

(12)
Proof

The starting point is the Gibbs variational principle, which relates the cumulant generating function and relative entropy (see dupuis2011weak ):

Proposition 2

If is measurable and then

(13)

If and , the result Eq. (12) follows from applying Eq. (13) to and bounding the supremum below by the value at . One can relax the assumptions to by truncating , using the result for bounded , and then taking limits. The case is trivial. ∎

The objects

(14)

appearing in the Gibbs information inequality, Eq. (12), have many interesting properties; we recall several of them below:

Proposition 3

Assume and . We have:

  1. (Divergence) is a divergence, i.e. and
    if and only if either or is constant -almost-surely (a.s.).

  2. (Linearization) If is sufficiently small we have

    (15)
  3. (Tightness) For consider . There exists with such that for any there exists a measure with

    (16)

    The measure has the form

    (17)

    where is the unique non-negative solution of .

Proof

Items 1 and 2 are proved in DKPP ; see also doi:10.1287/moor.2015.0776 for item 2. Various versions of the proof of item 3 can be found in in chowdhary_dupuis_2013 or DKPP . See Proposition 3 in DKPR for a more detailed statement of the result; see also similar results in BREUER20131552 ; doi:10.1111/mafi.12050 . ∎

3 Robustness of QoIs via Goal Oriented Information Theory

Much of the utility of the Gibbs information inequality, as expressed in Eq. (12), comes from the manner in which it splits the problem of obtaining UQ bounds into two sub-problems:

  1. Bound the cumulant generating function of the observable with respect to the base process, . (Note that there is no dependence on the alternative measure, .)

  2. Bound the relative entropy of the alternative model with respect to the base model: .

Despite the tightness property given in Theorem 3, when one is interested in a specific QoI and a specific (class of) alternative measures, the Gibbs information inequality, Eq. (12), is not quite ideal. Requiring a bound on the full relative entropy of the alternative model with respect to the base, , is too restrictive in general and can lead to poor or useless bounds, e.g. when and are the distributions of time-dependent processes on path space with infinite time-horizon. In the following theorem, we show how a simple modification results in a relative-entropy term targeted at the information present in the QoI:

Theorem 3.1 (Goal-Oriented Information Inequality)

Let .

  1. If . Then

    (18)

    where denotes the distribution of on (or simply on , if is real-valued) under , and similarly for .

  2. If is -measurable, where is a sigma sub-algebra, then

    (19)
Remark 1

Eq. (19) is one of the key properties that makes the bound Eq. (18) useful in practice. The objective is to find the smallest such for which is ‘easy’ to compute/bound. This will be made more concrete in Theorem 3.2 below.

Due to the way Eq. (18) and Eq. (19) allow for bounds to be tailored to the QoI, , we call any such a goal-oriented relative entropy associated with and refer to the general approach as goal-oriented information theory.

Proof

To prove the theorem, first write

(20)

and then use Proposition 1 to obtain

(21)

Here, denotes the identity function on .

Recalling the definition of , Eq. (10), we find

(22)

If is -measurable for some sigma sub-algebra, , then Eq. (19) is a variant of the data processing inequality (see, for example, 1705001 ). ∎

It is also useful to have an uncentered version of Theorem 3.1:

Corollary 1

Let , . Then

(23)
Remark 2

In many cases, Corollary 1 can be used to prove that , by first truncating , so that Eq. (23) can be applied to a bounded QoI, and then taking limits using the dominated and/or monotone convergence theorems. Such a truncation argument is also how one obtains Corollary 1, in its full generality, from Theorem 3.1.

Eq. (23) is also useful when the QoI for the alternative model is different from that of the base model; in such cases, working with the -centered QoI, , is inconvenient. This situation arises in the example in Section 5.5.

The relative entropy term in Eq. (18) is generally inaccessible, but Eq. (19) gives one a great deal of freedom to relax the UQ bound, with the intent of finding a computable upper bound. More specifically, our goal will be to find (the smallest) for which we have a -measurable function, , that satisfies . Given this, the following theorem provides robustness bounds on the QoI that are expressed solely in terms of expectations under the base model, . As we are thinking of as the tractable model, these UQ bounds are often computable.

Theorem 3.2

Let , be -measurable, where is a sigma sub-algebra, and suppose

(24)

for some -measurable . Then

(25)
Proof

Given , apply Corollary 1, the data processing inequality, and assumption Eq. (24) to the -measurable QoI, :

(26)

In particular, bounding the right-hand-side by the value at , we can cancel the term (which is assumed to be finite) to find

(27)

Taking the infimum over all yields the result. ∎

Remark 3

Assuming that can be differentiated under the integral, we see that taking in Eq. (25) results in an equality. This is rarely a practical choice, but it illustrates that Eq. (25) is tight for appropriately chosen and .

Once a satisfying the hypotheses of Theorem 3.2 has been identified, one way to search for improved bounds is to take any other -measurable, -mean zero function, , replace with in Eq. (25), and minimize over .

All further UQ bounds in this paper will be based upon the goal-oriented information inequality, as expressed by one of Theorem 3.1, Corollary 1, or Theorem 3.2. To use these results, the general workflow is as follows:

  1. Given a QoI , a base model , and an alternative model, , first identify a sigma sub-algebra, , with respect to which is measurable. should be as small as possible, while keeping in mind that the subsequent steps of this procedure must be feasible as well.

  2. Identify a -measurable function, , that satisfies . Again, one wants a tight bound, but must also be simple enough to make following steps tractable.

  3. Obtain robustness bounds that don’t involve , either by using Theorem 3.2, or by separately bound (see Section 4.1), and then substituting the result into the goal-oriented information inequality, Theorem 3.1.

  4. Bound the resulting -expectations. Depending on the choices made above, this may or may not be tractable.

    In the examples we treat in Section 5, these (moment generating functions) will be exactly computable. For the case of bounded , one can also utilize more generic inequalities, such as the Hoeffding, Bernstein, and Bennett inequalities (see DemboZeitouni ; BLM ), which yield bounds on the MGF in terms of the

    -norm and variance of the QoI under

    .

  5. Solve the resulting optimization problem; the optimization is over a 1-D parameter, and is typically simple to perform numerically.

Even with the above outline, it is not always transparent how each step is to be accomplished in any given problem. A general framework for time-dependent QoI’s up to a stopping time, our primary area of application, will be given in the following subsection, with more specific classes of QoI’s being addressed in Section 4.

3.1 UQ for QoI’s up to a Stopping Time

Our applications of the above general theory will be to time-dependent QoI’s up to a stopping time. Many important examples fit under this umbrella, such as time averages of observables, discounted observables, and the expectation and distribution of stopping times; see Section 4 for details on these applications. First we discuss the general framework.

We will work with QoI’s of the following form:

Assumption 1

Suppose:

  1. is a filtered probability space, where , or . We define .

  2. and are probability measures on .

  3. , written , is progressively measurable (progressive for short) i.e. is measurable and is -measurable for all (intervals refer to subsets of ).

  4. is a -stopping time.

The quantity-of-interest we consider is the stopped process, . First recall:

Lemma 1

is -measurable, where

(28)

is the sigma-algebra up to the stopping-time .

For a general stopped QoI, , we have the following centered UQ bounds, following from Theorem 3.1, with the choice :

Corollary 2

Suppose . Then

(29)

One similarly has an uncentered variant:

Corollary 3

Suppose . Then

(30)

From these results, we see that for stopped QoI’s, is an appropriate goal-oriented relative entropy.

4 Implementation of Robustness Bounds for Several Classes of QoIs

Next, we apply the above framework to derive more specialized UQ bounds for several classes of problems, essentially carrying out the steps of the ‘user’s guide’ from Section 3 for the QoI’s in Table 1. Not all of our examples will fit neatly into these categories, but they do give one an idea of how the method can be used. Consideration of specific models can be found in Section 5.

4.1 Bounding the Relative Entropy

First, we show how the above results can be used to bound a goal-oriented relative entropy. First consider the generic case of a QoI, , that is -measurable for some sigma sub-algebra .

If then

(31)

One can now attempt to bound the relative entropy by applying our UQ framework to the QoI, . More generally, we obtain the following as a consequence of Theorem 3.2:

Corollary 4

Suppose , is -measurable for some sigma sub-algebra and . Then

(32)

When the QoI is a stopped process, , and one takes , then applying this result requires one to find a progressive process, , such that

(33)

This can often be done for Markov processes; see Appendix B.

Remark 4

The utility of the above result is primarily in situations when the UQ bound Eq. (25) from Theorem 3.2 is still difficult to compute. In such cases, it can be useful to first bound via Eq. (32) and then use the UQ bound from Theorem 3.1.

Often, a useful can be obtained from by eliminated a -mean-zero component; see Appendix B for examples.

4.2 Distribution of a Stopping-Time

Next we show how to bound the distribution of a stopping time. First, consider the general case of a QoI of the form , where is some event. The cumulant generating function is

(34)

Theorem 3.1 then gives the following UQ bound:

Corollary 5

Let be a sigma sub-algebra of and . Then

(35)

In particular, given a stopping time, , and , Corollary 5 can be applied to and , resulting in a bound on the distribution of .

Remark 5

Given (a numerical estimate of) the single number (i.e. base-model probability) and a bound on the relative entropy (see the methods of Appendix B), the 1-D optimization problem in Eq. (5) is trivial to solve numerically. This can make Eq. (5) useful even when the base model is intractable analytically, as long as it is inexpensive to compute via simulation.

Remark 6

UQ bounds based on Corollary 5 are far from optimal for very rare events. See DKPR ; doi:10.1137/130939730 for a related approach to UQ using Rényi divergences instead of relative entropy.

4.3 Expectation of a Stopping Time

Here we consider the expectation of a stopping time, . For , Theorem 2 applied to yields

(36)

For unbounded , it is often useful to use Lemma 4 from Appendix B to obtain:

(37)

In addition, if a relative entropy bound of the form

(38)

holds for all (see Appendix B for common classes of examples where this is the case), then one can use Theorem 3.2 with and . Taking via the monotone and dominated convergence theorems, results in:

Corollary 6

Assume the relative entropy satisfies a bound of the form Eq. (38) for all . Then

(39)

We illustrate this technique on several concrete examples in Section 5.

Remark 7

One does not need to assume to derive Eq. (39); if the given upper bound is proven to be finite, then that is sufficient to prove that is finite.

4.4 Time-Integral/Discounted QoI’s

A particular sub-case of interest is that of time-dependent QoI’s, , that are themselves the time-integral of some other process; here we derive alternative (uncentered) UQ bounds that are applicable in such circumstances. A centered version also holds, with obvious modifications.

Theorem 4.1

Let be a sigma-finite positive measure on and be progressively measurable such that exists (in ) for all , for all , . For define the right-continuous progressive process

(40)

Suppose we also have one of the following two conditions:

  1. is a finite measure and .

  2. (41)

Then

(42)
Proof

First suppose Eq. (41) holds. This allows for use of Fubini’s theorem:

(43)

is -measurable, hence Corollary 1 implies, for -a.e. ,

(44)

Combining these gives the result.

If is finite and is non-negative then repeat the above calculations for and take , using the dominated and monotone convergence theorems. ∎

Remark 8

If is a probability measure then one could alternatively apply Theorem 3.1 to the product measures and . However, Jensen’s inequality implies that the bound Eq. (4.1) is tighter than the bound obtained by this alternative procedure.

Remark 9

When , if one attempts to use Theorem 2 in place of Theorem 4.1 then one generally finds that the relative-entropy term, , is infinite and so the corresponding UQ bound is useless. On the other hand, the bound Eq. (4.1) often yields something nontrivial; see the example in Section 5.4 below.

Theorem Eq. (4.1) is especially useful for studying exponentially discounted QoI’s, where for some and . Such QoIs are often used in control theory (see page 147 in peter2014uncertainty ) and economics (see page 64 in page2013applications ).

In the exponentially-discounted case, the bound becomes

(45)

One can weaken either of the bounds Eq. (4.1) and Eq. (45) by pulling the infimum over outside the integral. The bound then consists of two terms, an integrated moment generating function, and an integrated relative entropy:

(46)

In the exponentially discounted case, the latter can be written

(47)

which is the same information-theoretic quantity that was introduced in peter2014uncertainty (see page 147) in the context of control theory: an exponentially discounted relative entropy. Theorem 4.1 provides a rigorous justification for its use in UQ of exponentially discounted QoI’s.

4.4.1 Time-Integral Processes with a Smooth Density

If the measure defining the integral QoI has a smooth density and the QoI is stopped at then one can take yet another approach.

Suppose that with , , , and as . The quantity of interest is then

(48)

For simplicity, we will assume here that is uniformly bounded.

Integrating by parts, we can rewrite as

(49)

where we define the probability measure and note that , since is assumed to be non-increasing and integrate to .

Using Fubini’s theorem and Eq. (53) we obtain

(50)

where and

(51)
Remark 10

The utility of the above approach is largely because it can lead to a connection between the cumulant-generating-function and the Feynman-Kac semigroup; this allows one to utilize functional inequalities (Poincaré, log-Sobolev, etc.) in order to bound the cumulant generating function; again, see 2018arXiv181205174B for details on this technique.

4.5 Time-Averages

Finally, we consider the case of a time-average of some observable: , , , , and

(52)

(One can similarly treat the discrete-time case.) UQ bounds for such QoI’s were previously treated in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . Here we show how they fit into the general framework of Section 3.1, and that we recover the same bounds.

Supposing that , Theorem 2 yields

(53)

where

(54)

and we reparameterized in the infimum to better exhibit the behavior as .

In particular, we see that an appropriate information-theoretic quantity for controlling the long-time behavior is the relative entropy rate:

(55)

which is finite in many cases of interest. This is the same result as obtained in DKPP by more specialized methods.

Remark 11

The time integral in the exponent of the expectation in Eq. (54) can again yield a connection to the Feynman-Kac semigroup, and the possibility of bounding it via functional inequalities 2018arXiv181205174B . This is one reason why we use Theorem