1. Introduction
Often, we find ourselves in a situation where we have to make some decision , which we may freely choose from a set of available decisions. Usually, we do not choose arbitrarily in : indeed, we wish to make a decision that performs best according to some criterion, i.e., an optimal decision. It is commonly assumed that each decision induces a realvalued gain : in that case, a decision is considered optimal in if it induces the highest gain among all decisions in . This holds for instance if each decision induces a lottery over some set of rewards, and these lotteries form an ordered set satisfying the axioms of von Neumann Morgenstern [1], or more generally, the axioms of for instance Herstein and Milnor [2], if we wish to account for unbounded gain.
So, we wish to identify the set of all decisions that induce the highest gain. Since, at this stage, there is no uncertainty regarding the gains , , the solution is simply
(1) 
Of course, may be empty; however, if the set is a compact subset of —this holds for instance if is finite—then contains at least one element. Secondly, note that even if contains more than one decision, all decisions in induce the same gain ; so, if, in the end, the gain is all that matters, it suffices to identify only one decision in —often, this greatly simplifies the analysis.
However, in many situations, the gains induced by decisions in
are influenced by variables whose values are uncertain. Assuming that these variables can be modelled through a random variable
that takes values in some set (the possibility space), it is customary to consider the gain as a socalled gamble on , that is, we view as a realvalued gain that is a bounded function of , and that is expressed in a fixed stateindependent utility scale. So, is a bounded –mapping, interpreted as an uncertain gain: taking decision , we receive an amount of utility when turns out to be the realisation of . For the sake of simplicity, we shall assume that the outcome of is independent of the decision we take: this is called actstate independence. What decision should we take?Irrespective of our beliefs about , a decision in is not optimal if its gain gamble is pointwise dominated by a gain gamble for some in , i.e., if there is an in such that for all and for at least one : choosing guarantees a higher gain than choosing , possibly strictly higher, regardless of the realisation of . So, as a first selection, let us remove all decisions from whose gain gambles are pointwise dominated (see Berger [3, Section 1.3.2, Definition 5 ff., p. 10]):
(2) 
where is understood to be pointwise, and is understood to be the negation of . The decisions in are called admissible, the other decisions in are called inadmissible. Note that we already recover Eq. (1) if there is no uncertainty regarding the gains , i.e., if all are constant functions of . When do admissible decisions exist? The set is nonempty if is a nonempty and weakly compact subset of the set of all gambles on (see Theorem 3 further on). Note that this condition is sufficient, but not necessary.
In what follows, we shall try to answer the following question: given additional information about , how can we further reduce the set of admissible decisions? The paper is structured as follows. Section 2 discusses the classical approach of maximising expected utility, and explains why it is not always a desirable criterion for selecting optimal decisions. Those problems are addressed in Section 3, discussing alternative approaches to deal with uncertainty and optimality, all of which attempt to overcome the issues raised in Section 2, and all of which are known from the literature. Finally, Section 4 compares these alternative approaches, and explains how optimal decisions can be obtained in a computationally efficient way. A few technical results are deferred to the appendix, where we, among other things, generalize a wellknown technical condition on the existence of optimal decisions.
2. Maximising Expected Utility?
In practice, beliefs about are often modelled by a (possibly finitely additive) probability measure on a field of subsets of , and one then arrives at a set of optimal decisions by maximising their expected utility with respect to ; see for instance Raiffa and Schlaifer [4, Section 1.1.4, p. 6], Levi [5, Section 4.8, p. 96, ll. 23–26], or Berger [3, Section 1.5.2, Paragraph I, p. 17]. Assuming that the field is sufficiently large such that the gains are measurable with respect to —this means that every is a uniform limit of simple gambles—the expected utility of the gain gambles is given by:
where we take for instance the Dunford integral on the right hand side; see Dunford [6, p. 443, Sect. 3], and Dunford and Schwartz [7, Part I, Chapter III, Definition 2.17, p. 112]—this linear integral extends the usual textbook integral (see for instance Kallenberg [8, Chapter 1]) to case where is not additive. Recall that we have assumed actstate independence: is independent of .
As far as it makes sense to rank decisions according to the expected utility of their gain gambles, we should maximise expected utility:
(3) 
When do optimal solutions exist? The set is guaranteed to be nonempty if is a nonempty and compact subset of the set of all gambles on , with respect to the supremum norm. Actually, this technical condition is sufficient for existence with regard to all of the optimality conditions we shall discuss further on. Therefore, without further ado, we shall assume that is nonempty and compact with respect to the supremum norm. A slightly weaker condition is assumed in Theorem 5, in the appendix of this paper.
Unfortunately, it may happen that our beliefs about cannot be modelled by a probability measure, simply because we have insufficient information to identify the probability of every event in . In such a situation, maximising expected utility usually fails to give an adequate representation of optimality.
For example, let be the unknown outcome of the tossing of a coin; say we only know that the outcome will be either heads or tails (so ), and that the probability of heads lays between and . Consider the decision set and the gain gambles
Clearly, , and
Concluding, if we have no additional information about , but still insist on using a particular (and necessarily arbitrary) , which is only required to satisfy , we find that is not very robust against changes in . This shows that maximising expected utility fails to give an adequate representation of optimality in case of ignorance about the precise value of .
3. Generalising to Imprecise Probabilities
Of course, if we have sufficient information such that can be identified, nothing is wrong with Eq. (3). We shall therefore try to generalise Eq. (3). In doing so, following Walley [9], we shall assume that our beliefs about are modelled by a realvalued mapping defined on a—possibly only very small—set of gambles, that represents our assessment of the lower expected utility for each gamble in ;^{1}^{1}1The upper expected utility of a gamble is if and only if the lower expected utility of is . So, for any gamble in , , and therefore, without loss of generality, we can restrict ourselves to lower expected utility. note that can be chosen empty if we are completely ignorant. Essentially, this means that instead of a single probability measure on , we now identify a closed convex set of finitely additive probability measures on , described by the linear inequalities
(4) 
We choose the domain of the measures sufficiently large such that all gambles of interest, in particular those in and the gain gambles , are measurable with respect to . Without loss of generality, we can assume to be the power set of , although in practice, it may be more convenient to choose a smaller field.
For a given measurable gamble , not necessarily in , we may also derive a lower expected utility by minimising subject to the above constraints, and an upper expected utility by maximising over the above constraints. In case and
are finite, this simply amounts to solving a linear program.
In the literature, is called a credal set (see for instance Giron and Rios [10], and Levi [5, Section 4.2, pp. 76–78], for more comments on this model), and is called a lower prevision (because they generalise the previsions, which are fair prices, of De Finetti [11, Vol. I, Section 3.1, pp. 69–75]).
The mapping obtained, corresponds exactly to the socalled natural extension of (to the set of measurable gambles), where is interpreted as a supremum buying price for (see Walley [9, Section 3.4.1, p. 136]). In this interpretation, for any , we are willing to pay any utility prior to observation of , if we are guaranteed to receive once turns out to be the outcome of . The natural extension then corresponds to the highest price we can obtain for an arbitrary gamble , taken into account the assessed prices for . Specifically,
(5) 
where varies over , over , , …, vary over , and , …, over .
It may happen that is empty, in which case is undefined (the supremum in Eq. (5) will always be ). This occurs exactly when incurs a sure loss as a lower prevision, that is, if we can find a finite collection of gambles , …, in such that , which means that we are willing to pay more for this collection than we can ever gain from it, which makes no sense of course.
Finally, it may happen that does not coincide with on . This points to a form of incoherence in : this situation occurs exactly when we can find a finite collection of gambles , , …, and nonnegative real numbers , …, , such that
This means that we can construct a price for , using the assessed prices for , which is strictly higher than . In this sense, corrects , as is apparent from Eq. (5).
Although the belief model described above is not the most general we may think of, it is sufficiently general to model both expected utility and complete ignorance: these two extremes are obtained by taking either equal to a singleton, or equal to the set of all finitely additive probability measures on (i.e., ). It also allows us to demonstrate the differences between different ways to make decisions with imprecise probabilities on the example we presented before.
In that example, the given information can be modelled by, say, a lower prevision on , defined by and , where is the gamble defined by and . For this , the set corresponds exactly to the set of all probability measures on , such that . We also easily find for any gamble on that
3.1. Maximin and Maximax
As a very simple way to generalise Eq. (3), we could take the lower expected utility as a replacement for the expected utility (see for instance Gilboa and Schmeidler [12], or Berger [3, Section 4.7.6, pp. 215–223]):
(6) 
this criterion is called maximin, and amounts to worstcase optimisation: we take a decision that maximises the worst expected gain. For example, if we consider the decision as a game against nature, who is assumed to choose a distribution in aimed at minimizing our expected gain, then the maximin solution is the best we can do. Applied on the example of Section 2, we find as a solution .
In case , i.e., in case of complete ignorance about , it holds that . Hence, in that case, maximin coincides with maximin (see Berger [3, Eq. (4.96), p. 216]), ranking decisions by the minimal (or infimum, to be more precise) value of their gain gambles.
Some authors consider bestcase optimisation, taking a decision that maximises the best expected gain (see for instance Satia and Lave [13]). In our example, the “maximax” solution is .
3.2. Maximality
Eq. (3) is essentially the result of pairwise preferences based on expected utility: defining the strict partial order on as whenever , or equivalently, whenever , we can simply write
where the operator selects the maximal, i.e., the undominated elements from a set with strict partial order .
Using the supremum buying price interpretation, it is easy to derive pairwise preferences from : define as whenever . Indeed, means that we are disposed to pay a strictly positive price in order to take decision instead of , which clearly indicates strict preference of over (see Walley [9, Sections 3.9.1–3.9.3, pp. 160–162]). Since is a strict partial order, we arrive at
(7) 
as another generalisation of Eq. (3), called maximality. Note that can also be viewed as a robustification of over in . Applied on the example of Section 2, we find as a solution.
Note that Walley [9, Sections 3.9.2, p. 161] has a slightly different definition: instead of working from the set of admissible decisions as in Eq. (7), Walley starts with ranking if or ( and ), and then selects those decisions from that are maximal with respect to this strict partial order. Using Theorem 3 from the appendix, it is easy to show that Walley’s definition of maximality coincides with the one given in Eq. (7) whenever the set is weakly compact. This is something we usually assume to ensure the existence of admissible elements; in particular, weak compactness is assumed in Theorem 5 (see appendix). The benefit of Eq. (7) over Walley’s definition is that Eq. (7) is easier to manage in the proofs in the appendix.
3.3. Interval Dominance
Another robustification of is the strict partial ordering defined by whenever ; this means that the interval is completely on the right hand side of the interval . The above ordering is therefore called interval dominance (see Zaffalon, Wesnes, and Petrini [14, Section 2.3.3, pp. 68–69] for a brief discussion and references).
(8) 
The resulting notion is weaker than maximality: applied on the example of Section 2, , which is strictly larger than .
3.4. EAdmissibility
In the example of Section 2, we have shown that may not be very robust against changes in . Robustifying against changes of in , we arrive at
(9) 
this provides another way to generalise Eq. (3). The above criterion selects those admissible decisions in that maximize expected utility with respect to at least one in ; i.e., they select the Eadmissible (see Good [15, p. 114, ll. 8–9], or Levi [5, Section 4.8, p. 96, ll. 8–20]) decisions among the admissible ones. We find for the example.
In case is defined on and for all , then every Eadmissible decision is also admissible, and hence, in that case, gives us exactly the set of Eadmissible options.
4. Which Is the Right One?
Evidently, it is hard to pinpoint the right choice. Instead, let us ask ourselves: what properties do we want our notion of optimality to satisfy? Let us summarise a few important guidelines.
Clearly, whatever notion of optimality, it seems reasonable to exclude inadmissible decisions. For ease of exposition, let’s assume that the inadmissible decisions have already been removed from , i.e., ; this implies in particular that gives us the set of Eadmissible decisions.
Now note that, in general, the following implications hold:
as is also demonstrated by our example. A proof is given in the appendix, Theorem 1.
Eadmissibility, maximality, and interval dominance have the nice property that the more determinate our beliefs (i.e., the smaller ), the smaller the set of optimal decisions. In contradistinction, maximin and maximax lack this property, and usually only select a single decision, even in case of complete ignorance. However, if we are only interested in the most pessimistic (or most optimistic) solution, disregarding other reasonable solutions, then maximin (or maximax) seems appropriate. Utkin and Augustin [16] have collected a number of nice algorithms for finding maximin and maximax solutions, and even mixtures of these two. Seidenfeld [17] has compared maximin to Eadmissibility, and argued against maximin in sequential decision problems.
If we do not settle for maximin (or maximax), should we choose Eadmissibility, maximality, or interval dominance? As already mentioned, interval dominance is weaker than maximality, so in general we will end up with a larger (and arguably too large) set of optimal options. Assuming the nonadmissible decisions have been weeded, a decision is not optimal in with respect to interval dominance if and only if
(10) 
Thus, if has elements, interval dominance requires us to calculate natural extensions, and make comparisons, whereas for maximality, by Eq. (7), we must calculate natural extensions, and perform comparisons—roughly speaking, each natural extension is a linear program in (size of ) variables and (size of ) constraints, or vice versa if we solve the dual program. So, comparing maximality and interval dominance, we face a tradeoff between computational speed and number of optimal options.
However, this also means that interval dominance is a means to speed up the calculation of maximal and Eadmissible decisions: because every maximal decision is also interval dominant, we can invoke interval dominance as a first computationally efficient step in eliminating nonoptimal decisions, if we eventually opt for maximality or Eadmissibility. Indeed, eliminating those decisions that satisfy Eq. (10), we will also eliminate those decisions that are neither maximal, nor Eadmissible.
Regarding sequential decision problems, we note that dynamic programming techniques cannot be used when using interval dominance (see De Cooman and Troffaes [18]), and therefore, since dynamic programming yields an exponential speedup, maximality and Eadmissibility are certainly preferred over interval dominance once dynamics enter the picture.
This leaves Eadmissibility and maximality. They are quite similar: they coincide on all decision sets that contain two decisions. In case we consider larger decision sets, they coincide if the set of gain gambles is convex (for instance, if we consider randomised decisions). As already mentioned, Eadmissibility is stronger than maximality, and also has some other advantages over maximality. For instance, , so, choosing decision with probability and decision with probability is preferred to decision . Therefore, we should perhaps not consider decision as optimal.
Eadmissibility is not vulnerable to such argument, since no Eadmissible decision can be dominated by randomized decisions: if for some it holds that for all , then also
for any convex combination of gain gambles, and hence, it also holds that
which means that no convex combination can dominate with respect to .
A powerful algorithm for calculating Eadmissible options has been recently suggested by Utkin and Augustin [16, pp. 356–357], and independently by Kikuti, Cozman, and de Campos [19, Sec. 3.4]. If has elements, finding all (pure) Eadmissible options requires us to solve linear programs in variables and constraints.
As we already noted, through convexification of the decision set, maximality and Eadmissibility coincide. Utkin and Augustin’s algorithm can also cope with this case, but now one has to consider in the worst case linear programs, and usually several less: the worst case only obtains if all options are Eadmissible. For instance, if there are only Eadmissible pure options, one has to consider only at most of those linear programs, and again, usually less.
In conclusion, the decision criterion to settle for in a particular application, depends at least on the goals of the decision maker (what properties should optimality satisfy?), and possibly also on the size and structure of the problem if computational issues arise.
Acknowledgements
I especially want to thank Teddy Seidenfeld for the many instructive discussions about maximality versus Eadmissibility. I also wish to thank two anonymous referees for their helpful comments. This paper has been supported by the Belgian American Educational Foundation. The scientific responsibility rests with its author.
Appendix A Proofs
This appendix is dedicated to proving the connections between the various optimality criteria, and existence results mentioned throughout the paper. In the whole appendix, we assume the following:
Recall, denotes some set of decisions, and every decision induces a gain gamble , where is the set of all gambles (bounded – mappings).
denotes a lower prevision, defined on a subset of . With we denote a field on such that all gain gambles and gambles in are measurable with respect to , i.e., are a uniform limit of simple gambles. could be for instance the power set of .
is assumed to avoid sure loss, and is its natural extension to the set of all measurable gambles. is the credal set representing , as defined in Section 3. We will make deliberate use of the properties of natural extension (for instance, superadditivity: , and hence also ). We refer to Walley [9, Sec. 2.6, p. 76, and Sec. 3.1.2, p. 123] for an overview and proof of these properties.
We use the symbol for an arbitrary finitely additive probability measure on , and denotes the Dunford integral with respect to . This integral is defined on (at least) the set of all measurable gambles.
a.1. Connections between Decision Criteria
Theorem 1.
The following relations hold.
Proof.
Let .
Suppose that is maximax in : maximises in . Since is the upper envelope of , and is weak* compact (see Walley [9, Sec. 3.6]), there is a in such that . But, , for every because is maximax. Thus, belongs to .
Suppose that : there is a in such that maximises in . But then, because is the lower envelope of , for all in . Hence, by Eq. (7) on p. 7, must be maximal.
Suppose that is maximal. Then, again by Eq. (7), for all in . But, , hence, also for all in , which means that belongs to .
Finally, suppose that is maximin: maximises in . But then for all in ; must be maximal. ∎
a.2. Existence
We first prove a technical but very useful lemma about the existence of optimal elements with respect to preorders; it’s an abstraction of a result proved by De Cooman and Troffaes [18]. Let’s start with a few definitions.
A preorder is simply a reflexive and transitive relation.
Let be any set, and let be any preorder on . An element of a subset of is called maximal in if, for all in , implies . The set of maximal elements is denoted by
(11) 
For any in , we also define the upset of relative to as
Lemma 2.
Let be a Hausdorff topological space. Let be any preorder on such that for any in , the set is closed. Then, for any nonempty compact subset of , the following statements hold.

For every in , the set is nonempty and compact.

The set of maximal elements of is nonempty.

For every in , there is a maximal element of such that .
Proof.
(i). Since is reflexive, it follows that , so is nonempty. Is it compact? Clearly, , so is the intersection of a compact set and a closed set, and therefore must be compact too.
(ii). Let be any subset of the nonempty compact set that is linearly ordered with respect to . If we can show that has an upper bound in with respect to , then we can infer from a version of Zorn’s lemma [20, (AC7), p. 144] (which also holds for preorders) that has a maximal element. Let then be an arbitrary finite subset of . We can assume without loss of generality that , and consequently . This implies that the intersection of these upsets is nonempty: the collection of compact and hence closed ( is Hausdorff) subsets of has the finite intersection property. Consequently, since is compact, the intersection is nonempty as well, and this is the set of upper bounds of in with respect to . So, by Zorn’s lemma, has a maximal element: is nonempty.
(iii). Combine (i) and (ii) to show that the nonempty compact set has a maximal element with respect to . It is then a trivial step to prove that is also maximal in : we must show that for any in , if , then . But, if , then also since by construction. Hence, , and since is maximal in , it follows that . ∎
The weak topology on is simply the topology of pointwise convergence. That is, a net in converges weakly to in if for all .
Theorem 3.
If is a nonempty and weakly compact set, then contains at least one admissible decision, and even more, for every decision in , there is an admissible decision in such that .
Proof.
It is easy to derive from Eq. (2) that
Hence, a decision is admissible in exactly when its gain gamble is maximal in . We must show that has maximal elements.
By Lemma 2, it suffices to prove that, for every , the set is closed with respect to the topology of pointwise convergence.
Let be a net in , and suppose that converges pointwise to : for every , . But, since for every , it must also hold that . Hence, . We have shown that every converging net in converges to a point in . Thus, is closed. This establishes the theorem. ∎
Let’s now introduce a slightly stronger topology on . This topology has no particular name in the literature, so let’s just call it the topology. It is determined by the following convergence.
Definition 4.
Say that a net in converges to in , if

for all (pointwise convergence), and

(convergence in norm).
This convergence induces a topology on : it turns
into a locally convex topological vector space, which also happens to be Hausdorff. A topological basis at
consists for instance of the convex setsfor , and for all . It has more open sets and more closed sets than the weak topology, but it has less compact sets than the weak topology. On the other hand, this topology is weaker than the supremum norm topology, so it has fewer open and closed sets, and more compact sets, compared to the supremum norm topology. Note that in case is finite, it reduces to the weak topology, which is in that case also equivalent to the supremum norm topology.
Note that , , and for all , are continuous, simply because
(see Walley [9, p. 77, Sec. 2.6.1(l)]). We will exploit this fact in the proof of the following theorem, generalising a result due to Walley [9, p. 161, Sec. 3.9.2].
Theorem 5.
If is nonempty and compact with respect to the topology, then the following statements hold.

is nonempty for all .

is nonempty.

is nonempty.

is nonempty.

is nonempty.

is nonempty.
Proof.
(i). Introduce the following order on : say that whenever . Let’s first show that, for all , the set is closed.
Let be a net in , and suppose that converges to . Since the integral is continuous, it follows that . Concluding, belongs to . We have established that every converging net in converges to a point in . Thus, is closed.
By Lemma 2, it follows that has at least one maximal element , that is, maximises in . Since any compact set is also weakly compact, there is a maximal element in such that , by Theorem 3. But then, , and hence, also maximises in . Because is maximal in , it also maximises in . This establishes that belongs to : this set is nonempty.
References
 [1] J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944.
 [2] I. N. Herstein, J. Milnor, An axiomatic approach to measurable utility, Econometrica 21 (2) (1953) 291–297.
 [3] J. O. Berger, Statistical Decision Theory and Bayesian Analysis, 2nd Edition, Springer, 1985.
 [4] H. Raiffa, R. Schlaifer, Applied Statistical Decision Theory, MIT Press, 1961.
 [5] I. Levi, The Enterprise of Knowledge. An Essay on Knowledge, Credal Probability, and Chance, MIT Press, Cambridge, 1983.
 [6] N. Dunford, Integration in general analysis, Transactions of the American Mathematical Society 37 (3) (1935) 441–453.
 [7] N. Dunford, J. T. Schwartz, Linear Operators, John Wiley & Sons, New York, 1957.
 [8] O. Kallenberg, Foundations of Modern Probability, 2nd Edition, Probability and Its Applications, Springer, 2002.
 [9] P. Walley, Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991.

[10]
F. J. Giron, S. Rios, QuasiBayesian behaviour: A more realistic approach to decision making?, in: J. M. Bernardo, J. H. DeGroot, D. V. Lindley, A. F. M. Smith (Eds.), Bayesian Statistics, University Press, Valencia, 1980, pp. 17–38.
 [11] B. De Finetti, Theory of Probability: A Critical Introductory Treatment, Wiley, New York, 1974–5, two volumes.
 [12] I. Gilboa, D. Schmeidler, Maxmin expected utility with nonunique prior, Journal of Mathematical Economics 18 (2) (1989) 141–153.
 [13] J. K. Satia, J. Roy E. Lave, Markovian decision processes with uncertain transition probabilities, Operations Research 21 (3) (1973) 728–740.

[14]
M. Zaffalon, K. Wesnes, O. Petrini, Reliable diagnoses of dementia by the naive credal classifier inferred from incomplete cognitive data, Artificial Intelligence in Medicine 29 (1–2) (2003) 61–79.
 [15] I. J. Good, Rational decisions, Journal of the Royal Statistical Society, Series B 14 (1) (1952) 107–114.
 [16] L. V. Utkin, T. Augustin, Powerful algorithms for decision making under partial prior information and general ambiguity attitudes, in: F. G. Cozman, R. Nau, T. Seidenfeld (Eds.), Proceedings of the Fourth International Symposium on Imprecise Probabilities and Their Applications, 2005, pp. 349–358.
 [17] T. Seidenfeld, A contrast between two decision rules for use with (convex) sets of probabilities: Gammamaximin versus Eadmissibility, Synthese 140 (1–2) (2004) 69–88.
 [18] G. de Cooman, M. C. M. Troffaes, Dynamic programming for deterministic discretetime systems with uncertain gain, International Journal of Approximate Reasoning 39 (2–3) (2004) 257–278.

[19]
D. Kikuti, F. G. Cozman, C. P. de Campos, Partially ordered preferences in decision trees: Computing strategies with imprecision in probabilities, in: R. Brafman, U. Junker (Eds.), Multidisciplinary IJCAI05 Workshop on Advances in Preference Handling, 2005, pp. 118–123.
 [20] E. Schechter, Handbook of Analysis and Its Foundations, Academic Press, San Diego, 1997.
Comments
There are no comments yet.