Variational-principle-based methods, relating expectations of a quantity of interest to information-theoretic divergences, have proven to be effective tools for obtaining robustness bounds, i.e. for uncertainty quantification (UQ), in the presence of both parametric and non-parametric model-form uncertainty chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . In the present work, we build upon these ideas by utilizing information-theoretic quantities, the so-called goal-oriented relative entropy and Fisher information, that are targeted at a specific quantity-of-interest (QoI) under investigation. Tailoring the method to the QoI results in tighter UQ bounds, and allows for the derivation of UQ bounds in cases where previous methods fail to apply.
In this paper, UQ will refer to the following general problem:
One has a baseline model, described by a probability measure, and a (family of) alternative model(s), . The base model is typically the ‘simpler’ model, meaning one can calculate QoI’s exactly, or it is relatively inexpensive to simulate, but it may contain many sources of error and uncertainty; it may depend on parameters with uncertain values (obtained from experiment, Monte-Carlo simulation, variational inference, etc.) or is obtained via some approximation procedure (dimension reduction, neglecting memory terms, linearization, asymptotic approximation, etc.). Any quantity of interest computed from therefore has significant uncertainty associated with it. In contrast, the family of alternative models is generally thought of as containing the ‘true’ (or at least, more precise) model, but due to its complexity or a lack of knowledge, it is considered intractable. Our UQ goal is then to, for a given QoI (real-valued function), , bound the error incurred from using in place of , i.e.
Here, and in the following, will denote the expectation under the probability measure . We will also refer to (resp. ) as the base model (resp. alternative model) QoI.
As examples, could be the failure-time of a system (or a more general stopping time), the value of some asset, or a cost-functional; a bound on Eq. (1) is then a performance guarantee, given the modeling uncertainty expressed by the family . See Section 5 for concrete examples.
The primary philosophy behind the methods derived below is that the bound Eq. (1) should be specialized to the QoI, , and the family
as much as possible, while still involving quantities that can be effectively computed/estimated. In short, one does not necessarily care aboutall differences between, and , but only the differences insofar as they impact the expectation of the QoI.
1.1 Summary of Results
Our new results all rest upon an improvement to the Gibbs-Variational-Principle-based UQ bounds that were developed and employed in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B ; this improvement is the content of Section 3. Specifically, Theorem 3.1 bounds (under suitable assumptions) the difference in expectation of a QoI, , with respect to a base model, , and an alternative model, , as follows:
Here is the QoI centered under , , is the cumulant generating function of under , denotes the distribution of on under , and similarly for , and
denotes the relative entropy (i.e. Kullback-Leibler divergence). See Section2 for detailed definitions and further background.
Again, is typically taken to be a tractable model, so that the cumulant generating function can be computed or bounded explicitly. As for the second term, is generally unobtainable, as it requires detailed knowledge of the distribution of under the intractable alternative model, . However, Eq. (2) becomes a practical tool for UQ when it is combined with the data processing inequality (again, see Theorem 3.1). Specifically, if is -measurable, where is a sigma sub-algebra of the full sigma algebra of the probability space, then
Here, denotes the restriction of the probability measure to the sigma sub-algebra. For example, many of our applications will involve some -stopping time, , in which case we take , the sigma-algebra up to the stopping time.
Eq. (3) gives one a great deal of freedom when weakening the robustness bound Eq. (2); ideally, this freedom can be exploited to obtain bounds that depend on an explicitly computable relative entropy, while still maintaining as much information regarding as possible (i.e. enlarging as little as possible). With this aim in mind, we call a goal-oriented relative entropy, to contrast it with the non-goal-oriented quantity . The practical benefit of using goal-oriented quantities will be demonstrated throughout our examples; for a particularly simple and clear demonstration, see Section 5.1.1, especially Figure 1.
Clearly, it is only useful to weaken the robustness bound Eq. (2) via the data processing inequality when is computable in some sense. In practice, we will require a bound of the following form to hold: for some -measurable . Given this, we obtain UQ bounds that only involve expectations under the (tractable) base model, (see Theorem 3.2):
In fact, (under appropriate assumptions) taking makes Eq. (4) an equality. Of course, this choice is often impractical to work with; one should aim to select appropriate and so that the right-hand-side of Eq. (4) is comparatively simple to compute/estimate, while at the same time keeping the inequalities as tight as possible. Apart from Eq. (4), other techniques for eliminating the -dependence of the bounds are also useful; see Remark 4 and the example in Section 5.5.2.
Eq. (2) and Eq. (4) are non-perturbative UQ-bounds. For parametric families, one can similarly obtain perturbative UQ-bounds (i.e. sensitivity bounds) in terms of so-called goal-oriented Fisher information; see Appendix A.
Our primary application of these results will be to QoI’s up to a stopping time. In this setting, one has a time-dependent process, , adapted to a filtration , and a -stopping time , and the goal is to bound the expectation of the stopped process, . This general framework is discussed in Section 3.1.
Bounding the relative entropy up to a stopping time.
Bounding the distribution of a stopping time.
Bounding the expected value of a stopping time.
Bounding the expectation of exponentially discounted observables.
Bounding the expectation of time averages. (These were treated previously in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . We show that our framework leads to the same bounds in this case.)
For example, in Corollary 6 we obtain the following bounds on the expectation of a stopping time, :
for any alternative model, , satisfying a relative entropy bound of the form
for some and all . As we will show, the bound Eq. (5) does not require a-priori knowledge that is finite.
Appendix B contains formulas for the relative entropy up to a stopping time for Markov processes in both discrete and continuous time. In particular, we will see that a bound of the form Eq. (6) is not uncommon when the base and alternative models come from Markov processes.
Finally, we illustrate the use of these results in Section 5 through several examples. Specifically, we study:
The distribution and expectation of hitting times of Brownian motion with constant drift, as compared to Brownian motion perturbed by a non-constant drift.
Expected hitting times of a Ornstein-Uhlenbeck process perturbed by a non-constant drift.
Exponentially-discounted cost for stochastic control systems.
Semi-Markov perturbations of a -queue.
The value of American put options in a variable-interest-rate environment.
2 Background on Uncertainty Quantification via Variational Principles
Here we record important background information on the variational-principle approach to UQ; the new methods presented in this paper will be built upon these foundations, starting in Section 3.
First, we fix some notation. Let be a probability measure on a measurable space .
will denote the extended reals and we will refer to a random variableas a quantity-of-interest (QoI). The cumulant generating function of is defined by
where denotes the natural logarithm. We use the continuous extensions of to and of to .
The following notation will be used for the set of real-valued QoIs with a well-defined and finite moment generating function (MGF) under:
It is not difficult to prove (see e.g. DemboZeitouni ) that for , the cumulant generating function is a convex function, finite and infinitely differentiable in some interval with , equal to outside of , and such have moments of all orders.
If we write
for the centered quantity of mean . We will often use the cumulant generating function for a centered observable :
Recall that .
Recall also that the relative entropy (or Kullback-Leibler divergence) of another probability measure, , with respect to is defined by
It has the property of a divergence, that is and if and only if ; see, for example, dupuis2011weak for this and further properties.
The starting point of our approach is a non-goal-oriented UQ bound, derived in chowdhary_dupuis_2013 ; DKPP (a similar inequality is used in the context of concentration inequalities, see e.g. BLM , and was also used independently in BREUER20131552 ; GlassermanXu ). We summarize the proof here for completeness.
Proposition 1 (Gibbs information inequality)
Let , . Then
The starting point is the Gibbs variational principle, which relates the cumulant generating function and relative entropy (see dupuis2011weak ):
If is measurable and then
appearing in the Gibbs information inequality, Eq. (12), have many interesting properties; we recall several of them below:
Assume and . We have:
(Divergence) is a divergence, i.e. and
if and only if either or is constant -almost-surely (a.s.).
(Linearization) If is sufficiently small we have
(Tightness) For consider . There exists with such that for any there exists a measure with
The measure has the form
where is the unique non-negative solution of .
Items 1 and 2 are proved in DKPP ; see also doi:10.1287/moor.2015.0776 for item 2. Various versions of the proof of item 3 can be found in in chowdhary_dupuis_2013 or DKPP . See Proposition 3 in DKPR for a more detailed statement of the result; see also similar results in BREUER20131552 ; doi:10.1111/mafi.12050 . ∎
3 Robustness of QoIs via Goal Oriented Information Theory
Much of the utility of the Gibbs information inequality, as expressed in Eq. (12), comes from the manner in which it splits the problem of obtaining UQ bounds into two sub-problems:
Bound the cumulant generating function of the observable with respect to the base process, . (Note that there is no dependence on the alternative measure, .)
Bound the relative entropy of the alternative model with respect to the base model: .
Despite the tightness property given in Theorem 3, when one is interested in a specific QoI and a specific (class of) alternative measures, the Gibbs information inequality, Eq. (12), is not quite ideal. Requiring a bound on the full relative entropy of the alternative model with respect to the base, , is too restrictive in general and can lead to poor or useless bounds, e.g. when and are the distributions of time-dependent processes on path space with infinite time-horizon. In the following theorem, we show how a simple modification results in a relative-entropy term targeted at the information present in the QoI:
Theorem 3.1 (Goal-Oriented Information Inequality)
If . Then
where denotes the distribution of on (or simply on , if is real-valued) under , and similarly for .
If is -measurable, where is a sigma sub-algebra, then
Eq. (19) is one of the key properties that makes the bound Eq. (18) useful in practice. The objective is to find the smallest such for which is ‘easy’ to compute/bound. This will be made more concrete in Theorem 3.2 below.
To prove the theorem, first write
and then use Proposition 1 to obtain
Here, denotes the identity function on .
Recalling the definition of , Eq. (10), we find
It is also useful to have an uncentered version of Theorem 3.1:
Let , . Then
In many cases, Corollary 1 can be used to prove that , by first truncating , so that Eq. (23) can be applied to a bounded QoI, and then taking limits using the dominated and/or monotone convergence theorems. Such a truncation argument is also how one obtains Corollary 1, in its full generality, from Theorem 3.1.
The relative entropy term in Eq. (18) is generally inaccessible, but Eq. (19) gives one a great deal of freedom to relax the UQ bound, with the intent of finding a computable upper bound. More specifically, our goal will be to find (the smallest) for which we have a -measurable function, , that satisfies . Given this, the following theorem provides robustness bounds on the QoI that are expressed solely in terms of expectations under the base model, . As we are thinking of as the tractable model, these UQ bounds are often computable.
Let , be -measurable, where is a sigma sub-algebra, and suppose
for some -measurable . Then
Assuming that can be differentiated under the integral, we see that taking in Eq. (25) results in an equality. This is rarely a practical choice, but it illustrates that Eq. (25) is tight for appropriately chosen and .
All further UQ bounds in this paper will be based upon the goal-oriented information inequality, as expressed by one of Theorem 3.1, Corollary 1, or Theorem 3.2. To use these results, the general workflow is as follows:
Given a QoI , a base model , and an alternative model, , first identify a sigma sub-algebra, , with respect to which is measurable. should be as small as possible, while keeping in mind that the subsequent steps of this procedure must be feasible as well.
Identify a -measurable function, , that satisfies . Again, one wants a tight bound, but must also be simple enough to make following steps tractable.
Bound the resulting -expectations. Depending on the choices made above, this may or may not be tractable.
In the examples we treat in Section 5, these (moment generating functions) will be exactly computable. For the case of bounded , one can also utilize more generic inequalities, such as the Hoeffding, Bernstein, and Bennett inequalities (see DemboZeitouni ; BLM ), which yield bounds on the MGF in terms of the
-norm and variance of the QoI under.
Solve the resulting optimization problem; the optimization is over a 1-D parameter, and is typically simple to perform numerically.
Even with the above outline, it is not always transparent how each step is to be accomplished in any given problem. A general framework for time-dependent QoI’s up to a stopping time, our primary area of application, will be given in the following subsection, with more specific classes of QoI’s being addressed in Section 4.
3.1 UQ for QoI’s up to a Stopping Time
Our applications of the above general theory will be to time-dependent QoI’s up to a stopping time. Many important examples fit under this umbrella, such as time averages of observables, discounted observables, and the expectation and distribution of stopping times; see Section 4 for details on these applications. First we discuss the general framework.
We will work with QoI’s of the following form:
is a filtered probability space, where , or . We define .
and are probability measures on .
, written , is progressively measurable (progressive for short) i.e. is measurable and is -measurable for all (intervals refer to subsets of ).
is a -stopping time.
The quantity-of-interest we consider is the stopped process, . First recall:
is -measurable, where
is the sigma-algebra up to the stopping-time .
For a general stopped QoI, , we have the following centered UQ bounds, following from Theorem 3.1, with the choice :
Suppose . Then
One similarly has an uncentered variant:
Suppose . Then
From these results, we see that for stopped QoI’s, is an appropriate goal-oriented relative entropy.
4 Implementation of Robustness Bounds for Several Classes of QoIs
Next, we apply the above framework to derive more specialized UQ bounds for several classes of problems, essentially carrying out the steps of the ‘user’s guide’ from Section 3 for the QoI’s in Table 1. Not all of our examples will fit neatly into these categories, but they do give one an idea of how the method can be used. Consideration of specific models can be found in Section 5.
4.1 Bounding the Relative Entropy
First, we show how the above results can be used to bound a goal-oriented relative entropy. First consider the generic case of a QoI, , that is -measurable for some sigma sub-algebra .
One can now attempt to bound the relative entropy by applying our UQ framework to the QoI, . More generally, we obtain the following as a consequence of Theorem 3.2:
Suppose , is -measurable for some sigma sub-algebra and . Then
When the QoI is a stopped process, , and one takes , then applying this result requires one to find a progressive process, , such that
This can often be done for Markov processes; see Appendix B.
The utility of the above result is primarily in situations when the UQ bound Eq. (25) from Theorem 3.2 is still difficult to compute. In such cases, it can be useful to first bound via Eq. (32) and then use the UQ bound from Theorem 3.1.
Often, a useful can be obtained from by eliminated a -mean-zero component; see Appendix B for examples.
4.2 Distribution of a Stopping-Time
Next we show how to bound the distribution of a stopping time. First, consider the general case of a QoI of the form , where is some event. The cumulant generating function is
Theorem 3.1 then gives the following UQ bound:
Let be a sigma sub-algebra of and . Then
In particular, given a stopping time, , and , Corollary 5 can be applied to and , resulting in a bound on the distribution of .
Given (a numerical estimate of) the single number (i.e. base-model probability) and a bound on the relative entropy (see the methods of Appendix B), the 1-D optimization problem in Eq. (5) is trivial to solve numerically. This can make Eq. (5) useful even when the base model is intractable analytically, as long as it is inexpensive to compute via simulation.
4.3 Expectation of a Stopping Time
Here we consider the expectation of a stopping time, . For , Theorem 2 applied to yields
In addition, if a relative entropy bound of the form
Assume the relative entropy satisfies a bound of the form Eq. (38) for all . Then
We illustrate this technique on several concrete examples in Section 5.
One does not need to assume to derive Eq. (39); if the given upper bound is proven to be finite, then that is sufficient to prove that is finite.
4.4 Time-Integral/Discounted QoI’s
A particular sub-case of interest is that of time-dependent QoI’s, , that are themselves the time-integral of some other process; here we derive alternative (uncentered) UQ bounds that are applicable in such circumstances. A centered version also holds, with obvious modifications.
Let be a sigma-finite positive measure on and be progressively measurable such that exists (in ) for all , for all , . For define the right-continuous progressive process
Suppose we also have one of the following two conditions:
is a finite measure and .
First suppose Eq. (41) holds. This allows for use of Fubini’s theorem:
is -measurable, hence Corollary 1 implies, for -a.e. ,
Combining these gives the result.
If is finite and is non-negative then repeat the above calculations for and take , using the dominated and monotone convergence theorems. ∎
Theorem Eq. (4.1) is especially useful for studying exponentially discounted QoI’s, where for some and . Such QoIs are often used in control theory (see page 147 in peter2014uncertainty ) and economics (see page 64 in page2013applications ).
In the exponentially-discounted case, the bound becomes
One can weaken either of the bounds Eq. (4.1) and Eq. (45) by pulling the infimum over outside the integral. The bound then consists of two terms, an integrated moment generating function, and an integrated relative entropy:
In the exponentially discounted case, the latter can be written
which is the same information-theoretic quantity that was introduced in peter2014uncertainty (see page 147) in the context of control theory: an exponentially discounted relative entropy. Theorem 4.1 provides a rigorous justification for its use in UQ of exponentially discounted QoI’s.
4.4.1 Time-Integral Processes with a Smooth Density
If the measure defining the integral QoI has a smooth density and the QoI is stopped at then one can take yet another approach.
Suppose that with , , , and as . The quantity of interest is then
For simplicity, we will assume here that is uniformly bounded.
Integrating by parts, we can rewrite as
where we define the probability measure and note that , since is assumed to be non-increasing and integrate to .
Using Fubini’s theorem and Eq. (53) we obtain
The utility of the above approach is largely because it can lead to a connection between the cumulant-generating-function and the Feynman-Kac semigroup; this allows one to utilize functional inequalities (Poincaré, log-Sobolev, etc.) in order to bound the cumulant generating function; again, see 2018arXiv181205174B for details on this technique.
Finally, we consider the case of a time-average of some observable: , , , , and
(One can similarly treat the discrete-time case.) UQ bounds for such QoI’s were previously treated in chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu ; doi:10.1137/130939730 ; 2018arXiv181205174B . Here we show how they fit into the general framework of Section 3.1, and that we recover the same bounds.
Supposing that , Theorem 2 yields
and we reparameterized in the infimum to better exhibit the behavior as .
In particular, we see that an appropriate information-theoretic quantity for controlling the long-time behavior is the relative entropy rate:
which is finite in many cases of interest. This is the same result as obtained in DKPP by more specialized methods.