 # Logical Stochastic Optimization

We present a logical framework to represent and reason about stochastic optimization problems based on probability answer set programming. This is established by allowing probability optimization aggregates, e.g., minimum and maximum in the language of probability answer set programming to allow minimization or maximization of some desired criteria under the probabilistic environments. We show the application of the proposed logical stochastic optimization framework under the probability answer set programming to two stages stochastic optimization problems with recourse.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Probability answer set programming [Saad and Pontelli2006, Saad2006, Saad2007a] is a declarative programming framework which aims to solve hard search problems in probability environments, and shown effective for probability knowledge representation and probability reasoning applications. It has been shown that many interesting probability reasoning problems are represented and solved by probability answer set programming, where probability answer sets describe the set of possible solutions to the problem. These probability reasoning problems include, but not limited to, reasoning about actions with probability effects and probability planning [Saad2007b]

, reinforcement learning in MDP environments

[Saad2008a], reinforcement learning in POMDP environments [Saad2011], contingent probability planning [Saad2009], and Bayesian reasoning [Saad2008b]. However, the unavailability of probability optimization aggregates, e.g., minimum and maximum in the language of probability answer set programming [Saad and Pontelli2006, Saad2006, Saad2007a] disallows the natural and concise representation of many interesting stochastic optimization problems that are based on minimization and maximization of some desired criteria imposed by the problem. The following stochastic optimization with recourse problem illuminates the need for these aggregates.

###### Example 1

Assume that a company produces some product, , and need to make a decision on the amount of units of to produce based on the market demand. The company made a decision on the amounts of units of product to produce at cost of $per unit of (first stage). However, market demand is stochastic with a discrete probability distribution and the market demand must be met in any scenario. The company can produce extra units of product to meet the market observed demands but with the cost of$ per unit (second stage). This means a recourse to extra production to meet the excess in demand. Assume that the probability distribution, , over market demand, , is given as follows where two scenarios are available, with and with . Formally, let be the number of units of product the company produces at the first stage and let , called recourse variable, be the number of units the company produces at the second stage to meet the market stochastic demand at scenario . The objective is to minimize the total expected cost. This two stages stochastic optimization problem is formalized as:

 minimize2x+I∑i=1pi(3yi)

subject to

 x+yi≥Dii=1,…,Ix≥0yi≥0i=1,…,I

where the constraint guarantee that demand is always met in any scenario and . The optimal solution to this two stages stochastic optimization with recourse problem is , , , and with minimum total expected cost equal to .

To represent this stochastic optimization problem in probability answer set programming and to provide correct solution to the problem, the probability answer set programming representation of the problem has to be able to represent the probability distributions of the problem domain and any probability distribution that may arise to the problem constraints along with the preference relation that minimizes or maximizes the objective function including the expected values that always appear in the objective functions of these types of stochastic optimization problems, and to be able to compare for the minimum or the maximum of the objective value across the generated probability answer sets.

However, the current syntax and semantics of probability answer set programming do not define probability preference relations or rank probability answer sets based on minimization or maximization of some desired criterion specified by the user. Therefore, in this paper we extend probability answer set programming with probability aggregate preferences to allow the ability to represent and reason and intuitively solve stochastic optimization problems. The proposed probability aggregates probability answer set optimization framework presented in this paper modifies and generalizes the classical aggregates classical answer set optimization presented in [Saad and Brewka2011] as well as the classical answer set optimization introduced in [Brewka et al.2003]. We show the application of probability aggregates probability answer set optimization to a two stages stochastic optimization with recourse problem described in Example (1), where a probability answer set program [Saad2007a]

(disjunctive hybrid probability logic program with probability answer set semantics) is used as probability answer sets generator rules.

## 2 Probability Aggregates Probability Answer Set Optimization

Probability answer set optimization programs are probability logic programs under the probability answer set semantics whose probability answer sets are ranked according to probability preference rules represented in the programs. A probability answer set optimization program, , is a pair of the form
, where is a union of two sets of probability logic rules and is a mapping, , associated to the set of probability logic rules . The first set of probability logic rules, , is called the generator rules that generate the probability answer sets that satisfy every probability logic rule in and the mapping associates to each atom, , appearing in , a disjunctive p-strategy that is used to combine the probability intervals obtained from different probability logic rules in with an atom appearing in their heads. is any set of probability logic rules with well-defined probability answer set semantics including normal, extended, and disjunctive hybrid probability logic rules [Saad and Pontelli2006, Saad2006, Saad2007a], as well as hybrid probability logic rules with probability aggregates (all are forms of probability answer set programming).

The second set of probability logic rules, , is called the probability preference rules, which are probability logic rules that represent the user’s probability quantitative and qualitative preferences over the probability answer sets generated by . The probability preference rules in are used to rank the generated probability answer sets from from the top preferred probability answer set to the least preferred probability answer set. Similar to [Brewka et al.2003], an advantage of probability answer set optimization programs is that and are independent. This makes probability preference elicitation easier and the whole approach is more intuitive and easy to use in practice.

In our introduction of probability answer set optimization programs, we focus on the syntax and semantics of the probability preference rules, , of the probability answer set optimization programs, since the syntax and semantics of the probability answer sets generator rules, , are the same as syntax and semantics of any set of probability logic rules with well-defined probability answer set semantics as described in [Saad and Pontelli2006, Saad2006, Saad2007a].

### 2.1 Basic Language

Let be a first-order language with finitely many predicate symbols, function symbols, constants, and infinitely many variables. A literal is either an atom in or the negation of an atom (), where is the Herbrand base of and is the classical negation. Non-monotonic negation or the negation as failure is denoted by . The Herbrand universe of is denoted by . Let be the set of all literals in , where . A probability annotation is a probability interval of the form , where are called probability annotation items. A probability annotation item is either a constant in (called probability annotation constant), a variable ranging over (called probability annotation variable), or (called probability annotation function) where is a representation of a monotone, antimonotone, or nonmonotone total or partial function and are probability annotation items.

Let be an arbitrary set of p-strategies, where () is the set of all conjunctive (disjunctive) p-strategies in . A hybrid literals is an expression of the form or , where are literals and and are p-strategies from . is the set of all ground hybrid literals formed using distinct literals from and p-strategies from . If is a hybrid literal is a probability annotation then is called a probability annotated hybrid literal.

A symbolic probability set is an expression of the form , where is a variable or a function term and , are probability annotation variables or probability annotation functions, and is a conjunction of probability annotated hybrid basic formulae. A ground probability set is a set of pairs of the form such that is a constant term and are probability annotation constants, and is a ground conjunction of probability annotated hybrid basic formulae. A symbolic probability set or ground probability set is called a probability set term. Let be a probability aggregate function symbol and be a probability set term, then is said a probability aggregate, where , , , , , , , , , , . If is a probability aggregate and is an interval , called guard, where are constants, variables or functions terms, then we say is a probability aggregate atom, where .

A probability optimization aggregate is an expression of the form , , , , , and , where is a probability aggregate function symbol and is a probability set term.

### 2.2 Probability Preference Rules Syntax

Let be a set of probability annotated hybrid literals, probability annotated probability aggregate atoms and probability optimization aggregates. A boolean combination over is a boolean formula over probability annotated hybrid literals, probability annotated probability aggregate atoms, and probability optimization aggregates in constructed by conjunction, disjunction, and non-monotonic negation (), where non-monotonic negation is combined only with probability annotated hybrid literals and probability annotated probability aggregate atoms

###### Definition 1

A probability preference rule, , over a set of probability annotated hybrid literals, probability annotated probability aggregate atoms and probability optimization aggregates, , is an expression of the form

 C1≻C2≻…≻Ck←Lk+1:μk+1,…,Lm:μm, notLm+1:μm+1,…,notLn:μn (1)

where are probability annotated hybrid literals and probability annotated probability aggregate atoms and are boolean combinations over .

Let and , where is a probability preference rule of the form (1). Intuitively, a probability preference rule, , of the form (1) means that any probability answer set that satisfies and is preferred over the probability answer sets that satisfy , some , but not , and any probability answer set that satisfies and is preferred over probability answer sets that satisfy , some , but neither nor , etc.

###### Definition 2

A probability answer set optimization program, , is a pair of the form , where is a set of probability logic rules with well-defined probability answer set semantics, the generator rules, is a set of probability preference rules, and is the mapping that associates to each literal, , appearing in a disjunctive p-strategy.

Let be a probability aggregate. A variable, , is a local variable to if and only if appears in and does not appear in the probability preference rule that contains . A global variable is a variable that is not a local variable. Therefore, the ground instantiation of a symbolic probability set

 S={X:[P1,P2]|C}

is the set of all ground pairs of the form , where is a substitution of every local variable appearing in to a constant from . A ground instantiation of a probability preference rule, , is the replacement of each global variable appearing in to a constant from , then followed by the ground instantiation of every symbolic probability set, , appearing in . The ground instantiation of a probability aggregates probability answer set optimization program, , is the set of all possible ground instantiations of every probability rule in .

###### Example 2

The two stages stochastic optimization with recourse problem presented in Example (1) can be represented as a probability aggregates probability answer set optimization program , where is any assignments of disjunctive p-strategies and is a set of disjunctive hybrid probability logic rules with probability answer set semantics [Saad2007a] of the form:

 domX(500)∨domX(550)∨domX(600)∨domX(650)∨domX(700).domY1(0):p1∨domY1(50):p1∨domY1(100):p1∨domY1(150):p1∨domY1(200):p1.domY2(0):p2∨domY2(50):p2∨domY2(100):p2∨domY2(150):p2∨domY2(200):p2.
 objective(X,Y1,Y2,2∗X+3∗p1∗Y1+3∗p2∗Y2)←domX(X),domY1(Y1):p1,domY1(Y2):p2.
 ←domX(X),domY1(Y1):p1,X+Y1<500.←domX(X),domY2(Y2):p2,X+Y2<700.

where and , , , are predicates represent the domains of possible values for the variables , , that represent the units of product corresponding to the variables described in Example (1), is a predicate that represents the objective value, , for the assignments of units of a product to the variables , , where is the expected cost for this assignment of variables.

The set of probability preference rules, , of consists of the probability preference rule

 minx{Cost:1|objective(X,Y1,Y2,Cost)}←

## 3 Probability Aggregates Probability Answer Set Optimization Semantics

Let denotes a set of objects. Then, we use to denote the set of all multisets over elements in . Let denotes the set of all closed intervals in , denotes the set of all real numbers, denotes the set of all natural numbers, and denotes the Herbrand universe. Let be a symbol that does not occur in . Therefore, the semantics of the probability aggregates are defined by the mappings:

• .

• .

• .

• .

• .

• .

• .

• .

• .

The application of and on the empty multiset return and respectively. The application of and on the empty multiset returns . The application of and on the empty multiset return and respectively. The application of on the empty multiset returns . However, the application of , , , on the empty multiset is undefined.

The semantics of probability aggregates and probability optimization aggregates in probability aggregates probability answer set optimization is defined with respect to a probability answer set, which is, in general, a total or partial mapping, , from to . In addition, the semantics of probability optimization aggregates , , , , , and are based on the semantics of the probability aggregates .

We say, a probability annotated hybrid literal, , is true (satisfied) with respect to a probability answer set, , if and only if . The negation of a probability hybrid literal, , is true (satisfied) with respect to if and only if or is undefined in . The evaluation of probability aggregates and the truth valuation of probability aggregate atoms with respect to probability answer sets are given as follows. Let be a ground probability aggregate and be a probability answer set. In addition, let be the multiset constructed from elements in , where is true w.r.t. . Then, the evaluation of with respect to is, , the result of the application of to , where if is not in the domain of and

### 3.1 Probability Preference Rules Semantics

In this section, we define the notion of satisfaction of probability preference rules with respect to probability answer sets. We consider that probability annotated probability aggregate atoms that involve probability aggregates from , , , , , are associated to the probability annotation .

Let be a ground probability aggregates probability answer set optimization program, be probability answer sets for (possibly partial), , , , , , and , , , , , and be a probability preference rule in . Then the satisfaction of a boolean combination, , appearing in , by is defined inductively as follows:

• satisfies iff .

• satisfies iff or is undefined in .

• satisfies iff and .

• satisfies iff or and .

• satisfies iff and and .

• satisfies iff or and or .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and or and .

• satisfies iff and for any , and and or and .

• satisfies iff and for any , and and or and .

• iff and .

• iff or .

The satisfaction of by is defined inductively as follows:

• satisfies iff .

• satisfies iff or is undefined in .

• satisfies iff and .

• satisfies iff or and .

• satisfies iff and and .

• satisfies iff or and or .

• satisfies iff , satisfies and , satisfies .

The application of the probability aggregates, , on a singleton , returns ( multiplied by ), i.e., . Therefore, we use and as abbreviations for the probability optimization aggregates and respectively, whenever is a singleton and . Similarly, the application of the probability aggregates, , on a singleton , returns , i.e., . Therefore, we use , , , , , and as abbreviations for the probability optimization aggregates , , , , , and respectively, whenever is a singleton and .

###### Definition 3

Let be a ground probability aggregates probability answer set optimization program, be a probability answer set for , and be a probability preference rule in , and be a boolean combination in . Then, we define the following notions of satisfaction of by :

• iff and .

• iff and does not satisfy any in .

• iff does not satisfy .

means that the body of and the boolean combination that appearing in the head of is satisfied by . However, means that is irrelevant (denoted by ) to , or, in other words, the probability preference rule is not satisfied by , because either one of two reasons. Either because the body of and non of the boolean combinations that appearing in the head of are satisfied by . Or because the body of is not satisfied by .

### 3.2 Probability Answer Sets Ranking

In this section we define the ranking of the probability answer sets with respect to a boolean combination, a probability preference rule, and with respect to a set of probability preference rules.

###### Definition 4

Let be a ground probability aggregates probability answer set optimization program, be two probability answer sets for , be a probability preference rule in , be boolean combination appearing in , and , , , , , and , , , , . Then, is strictly preferred over w.r.t. , denoted by , iff and or and (except is a probability optimization aggregate) and one of the following holds:

• implies iff .

• implies iff or is undefined in but defined .

• implies iff , , and .

• implies iff

• and or

• , , and

• implies iff , , and .

• implies iff

• and or

• , , and

• implies iff and .

• implies iff there exists such that and for all other , we have .

• implies iff there exists such that and for all other , we have