A distributed (preconditioned) projected-reflected-gradient algorithm for stochastic generalized Nash equilibrium problems

03/20/2020 ∙ by Barbara Franci, et al. ∙ Delft University of Technology 0

We consider the stochastic generalized Nash equilibrium problem (SGNEP) with joint feasibility constraints and expected-value cost functions. We propose a distributed stochastic preconditioned projected reflected gradient algorithm and show its almost sure convergence when the pseudogradient mapping is cocoercive. The algorithm is based on monotone operator splitting methods for SGNEPs when the expected-value pseudogradient mapping is approximated at each iteration via an increasing number of samples of the random variable, an approach known as sample average approximation. Finally, we show that a non-preconditioned variant of our proposed algorithm has less restrictive convergence guarantees than state-of-the-art (preconditioned) forward-backward algorithms.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Stochastic generalized Nash equilibrium problems (SGNEPs) have received some attention from the system and control community [1, 2, 3]. In a SGNEP, a set of agents interacts with the aim of minimizing their expected-value cost functions while subject to some joint feasibility constraints. The main feature is that both the cost function and the constraints are uncertain and depend on the strategies chosen by the other agents. Stochastic equilibrium problems arise when there is some uncertainty, modelled via a random variable with an unknown distribution. One main reason for the interest is related to their possible applications. For instance, any networked Cournot game with market capacity constraints and uncertainty in the demand can be modelled as a SGNEP [4, 5]. Other applications can be found in transportation systems, where the drivers perception of travel time is a possible source of uncertainty [6], and in electricity markets where companies schedule their energy production without fully knowing the demand [7].

Now, if the random variable is known, the expected–value formulation can be solved via a standard technique for deterministic GNEPs [8, 9]. Similarly to the deterministic case, one possible approach for SGNEPs is to recast the problem as a stochastic variational inequality (SVI) through the use of the Karush–Kuhn–Tucker conditions. Then, the problem can be rewritten as a monotone inclusion and solved via operator splitting techniques. Besides the fact that we use the algorithm to find a SGNE, the difficulty in the stochastic case is that the pseudogradient is usually not directly accessible, for instance because the expected value is hard or expensive to compute. For this reason, in many situations, the search for a solution of a SVI relies on samples of the random variable.

Two are the main approximation schemes used in the literature: the stochastic approximation (SA) and the sample average approximation (SAA). In the first case, the approximation is done by using only one realization of the random variable. SA was first presented in [10], it is computationally less expensive than SAA but usually it requires stronger assumptions on the pseudogradient mapping and on the parameters [2, 11, 12]. The second approximation scheme takes instead an increasing number of samples at every iteration. SAA has the disadvantage of being computationally costly but it requires weaker assumptions to ensure convergence [13, 14].

Depending on the monotonicity assumptions on the pseudogradient mapping or the affordable computational complexity, there are different algorithms that can be used to find a SGNE. Among others, one can consider the stochastic preconditioned forward–backward (SpFB) algorithm [15] which is guaranteed to converge to a Nash equilibrium under cocoercivity of the pseudogradient and by demanding one projection step per iteration. However, cocoercivity is not the weakest possible assumption, therefore one would like to have an algorithm that converges under mere monotonicity. For instance, the stochastic forward–backward–forward (SFBF) algorithm involves only one projection step per iteration but two costly evaluation of the pseudogradient mapping [13]. Another alternative is the stochastic extragradient (SEG) algorithm whose iterates are characterized by two projection steps and two evaluation of the pseudogradient mapping which may be expensive [14]. To summarize, having weaker assumptions comes at the price of implementing computationally expensive algorithms.

In this paper, we propose a stochastic preconditioned projected reflected gradient (SpPRG) algorithm for SGNEPs. The basic, deterministic version of this algorithm was first presented for variational inequalities by Malitsky in [16] and then extended to the stochastic case by Cui and Shanbhag in [11, 17]. Here, we consider the algorithm in [17] that uses the SAA scheme. The convergence of the algorithm is guaranteed when the pseudogradient mapping is monotone and “weak sharp”, a property that we discuss in Section V. Unfortunately, the latter property is not trivial to check on the problem data. Therefore, to cope with SGNEPs, we assume that the pseudogradient mapping is cocoercive. Furthermore, in order to make our algorithm distributed, we exploit a suitable preconditioning. We also show that if the equilibrium solution is unique, then mere monotonicity (as opposed to cocoercivity) is sufficient for convergence and preconditioning is not required. We enphasize that this is the first time that the PRG algorithm is designed for SGNEPs. Remarkably, under uniqueness of the solution, our algorithm has convergence guaranteed also for merely monotone pseudogradient mappings. This is a significant advantage compared to the SpFB which may not converge in that case. See Section VII for an example.

Notation: denotes the standard inner product and is the associated euclidean norm. We indicate that a matrix is positive definite, i.e., , with . indicates the Kronecker product between and . Given a symmetric , the -induced inner product is and the associated norm is defined as .

indicates the vector with

entries all equal to . Given . is the resolvent of the operator where is the identity operator. For a closed set the mapping denotes the projection onto , i.e., . is the indicator function of the set C, that is, if and otherwise. The set-valued mapping denotes the normal cone operator of the set , i.e., if otherwise.

For all , a mapping is: pseudomonotone if ; monotone if [18, Def. 2.3.1]; maximally monotone if there exists no monotone operator such that the graph of properly contains the graph of [19, Def. 20.20]; -cocoercive if for [18, Def. 2.3.9]; -Lipschitz continuous if for [19, Def. 1.47].

Ii Stochastic generalized Nash equilibrium problem

Ii-a Problem Setup

We consider a set of agents , each of them choosing its decision variable from its local decision set . The aim of each agent is to minimize its local cost function defined as

for some measurable function and . We note that the cost function depends on the local variable , on the decisions of the other agents and on the random variable . Specifically, the random variable

expresses the fact that there is some uncertainty in the cost function, given the associated probability space

. represents the mathematical expectation with respect to the distribution of 111For brevity, we use instead of , , and instead of .. We assume that is well defined for all the feasible .

The cost function should satisfy some assumptions, postulated next, to make our analysis possible. Such assumptions are standard for (stochastic) Nash equilibrium problems [1].

Assumption 1

For each and the function is convex and continuously differentiable. For each and , the function is convex, Lipschitz continuous, and continuously differentiable. The function is measurable and for each , its Lipschitz constant is integrable in

Since we consider a SGNEP, we introduce affine shared constraints, . Thus, we denote each agent feasible decision set with the set-valued mapping

(1)

where each indicates how agent is involved in the coupling constraints and . The collective feasible set can be then written as

(2)

where and . We suppose that there is no uncertainty in the constraints.

Standard assumptions for the constraints sets are postulated next [20].

Assumption 2

For each the set is nonempty, compact and convex. The set satisfies Slater’s constraint qualification.

The aim of each agent , given the decision variables of the other agents , is to choose a decision , that solves its local optimization problem, i.e.,

(3)

From a game-theoretic perspective, we aim at computing a stochastic generalized Nash equilibrium (SGNE) [1].

Definition 1

A collective variable is a stochastic generalized Nash equilibrium if, for all :

In other words, a SGNE is a vector of strategies where none of the agents can decrease its cost function by unilaterally deviating from its decision variable.

When Assumptions 1-2 hold, existence of a SGNE of the game in (3) is guaranteed by [1, §3.1] but uniqueness does not hold in general [1, §3.2].

Among all the possible Nash equilibria, we focus on those that are also solutions of an associated (stochastic) variational inequality. First let us define the pseudogradient mapping as

(4)

where the exchange between the expected value and the gradient is possible because of Assumption 1 [1, Lem. 3.4]. Then, the stochastic variational inequality is the problem of finding such that

(5)

with as in (4) and as in (1). We also note that any solution of is a SGNE of the game in (3) while the opposite does not hold in general. In fact, a game may have a Nash equilibrium while the associated VI may have no solution [21, Prop. 12.7].

Remark 1

Under Assumptions 1-2, the solution set of is non empty and compact, i.e. [18, Corollary 2.2.5].

Ii-B Operator-theoretic characterization

In this section, we rewrite the SGNEP as a monotone inclusion, i.e., the problem of finding a zero of a set-valued monotone operator.

To this aim, we characterize the SGNE of the game in terms of the Karush–Kuhn–Tucker (KKT) conditions for the coupled optimization problems in (3). For each agent , let us denote with the dual variable associated with the coupling constraints. Then, the Lagrangian function, for every , is given by It holds that the set of strategies is a SGNE if and only if the following KKT conditions are satisfied [22, Th. 4.6]:

(6)

Similarly, a variational SGNE (v-SGNE) can be characterized by using the KKT conditions associated to the SVI in (5) [22, Proposition 1.2.1]. The associated KKT optimality conditions reads as

(7)

The connection between the KKT conditions in (7) and a v-SGNE is summarized next.

From [20, Th. 3.1], it follows that if is a solution of at which the KKT conditions (7) hold, then is a solution of the SGNEP at which the KKT conditions (6) hold with . Viceversa, if is a solution of the SGNEP at which KKT conditions (6) hold with , is a solution of in (5). In words, [20, Th. 3.1] says that variational equilibria are those such that the shared constraints have the same dual variable for all the agents.

Iii Distributed stochastic preconditioned projected reflected gradient algorithm

In this section, we propose a distributed stochastic preconditioned projected reflected gradient (SpPRG) algorithm for finding a v-SGNE of the game in (3). The iterations are presented in Algorithm 1 which is inspired by [17, 16]. For each agent , the variables , and denote the local variables , and at the iteration time while , and are the step sizes. We note that agents can equivalently share the already computed dual variable or only and let the receiving agent do the computation. In any case, the number of times that the agents communicate is the same.

Initialization: and
Iteration : Agent
(1) Updates

(2): Receives for and for then updates:

(3): Receives for and , for all then updates:

Algorithm 1 Distributed Stochastic Forward–Backward

Since we want the algorithm to be distributed, we assume that each agent only knows its local data, i.e., , and . Moreover, each player is able to compute , given the collective decision . Since the expected value can be hard to evaluate, users compute an approximation (we give more details later on). We assume therefore that each agent has access to all the decision variables that affect its pseudogradient. These information are collected, for each agent , in the set , that is, the set of agents whose decision explicitly influences .

Since the v-SGNE requires consensus of the dual variables, we introduce an auxiliary variable for all . The role of is to help reaching consensus and it will be properly defined later in this section. The auxiliary variable and a local copy of the dual variable are shared through the graph . The set of edges represents the exchange of the private information on the dual variables: if agent can receive from agent . The set of neighbours of in is indicated with [8, 3]. Since each agent feasible set implicitly depends on all the other agents decisions (through the shared constraints), to reach consensus of the dual variables, all agents must coordinate and therefore, must be connected.

Assumption 3

The dual-variable communication graph is undirected and connected.

The weighted adjacency matrix of the dual variables graph is indicated with . Let be the Laplacian matrix associated to , where is the diagonal matrix of the agents degrees . It follows from Assumption 3 that the adjacency matrix and the Laplacian are both symmetric, i.e., and .

To obtain the iterations presented in Algorithm 1, let us rewrite the KKT conditions in (7) as

(8)

where is a set-valued mapping.

We note that the mapping can be written as the sum of two operators:

(9)

Then, finding a solution of the game in (3) translates in finding a zero of the operator or equivalently, .

Let be the Laplacian matrix of and set . Following [8], to force consensus on the dual variables, we impose the Laplacian constraint . Then, to preserve monotonicity, we augment the two operators and introducing the auxiliary variable . Let and and similarly let us define of suitable dimensions. Then, the extended operators read as

(10)

From now on, we indicate . The following lemma ensure that the zeros of are v-SGNE.

Lemma 1

Let Assumptions 1-3 hold. Consider the operators and in (9) and and in (10). Then, the following statements hold.

  • If , then is a v-SGNE of game in (3), i.e., solves the in (5). Moreover and satisfy the KKT condition in (7) i.e.,

  • and

It follows from [8, Th. 2].

The properties of the operators and in (10) depends on the properties of the mapping in (4).

Assumption 4

is -cocoercive for some .

Remark 2

If a function is -cocoercive, it is also -Lipschitz continuous [19, Remark 4.15].

A technical discussion on this assumption is postponed to Section V.

We can now show the necessary monotonicity properties of the extended operators.

Lemma 2

Let Assumptions 1 and 4 hold and let . Then, and in (10) have the following properties.

  • is -cocoercive where and is the maximum weighted degree of ;

  • is maximally monotone;

  • is -cocoercive, with ;

  • is maximally monotone.

It follows from [8, Lem. 5] and [8, Lem. 7].

Since the expected value can be hard to compute, we take an approximation. At this stage, it is not important to specify if we use sample average or stochastic approximation, therefore, in what follows, we replace in (4) with an approximation , given a vector sample of the random variable , and with

(11)

Given in (11) and in (10), we can write Algorithm 1 in compact form as

(12)

where is the preconditioning matrix. Specifically, let and similarly and of suitable dimensions. Then, we have

(13)

By expanding (12) with as in (11), as in (10) and as in (13), we obtain the iterations in Algorithm 1. We note that, since is lower block triangular, the iterations of Algorithm 1 are sequential, i.e., depends on the last update and of the agents strategies and of the auxiliary variable respectively.

Iv Convergence analysis with sample average approximation

Since the distribution of the random variable is unknown, in the algorithm we have replaced the expected value with its approximation . For the convergence analysis, we use the sample average approximation (SAA) scheme. We assume to have access to a pool of i.i.d. samples of the random variable collected, for all and for each agent , in the vectors . At each time , the approximation is

(14)

where is the batch size, i.e., the number of sample to be taken. We define the distance between the expected value and its approximation as

Since there is no uncertainty in the constraints, we have

where is the operator with approximation as in (14). Let us introduce the filtration , i.e., a family of -algebras such that , for all , and for all . The filtration collects the informations that each agent has at the beginning of each iteration . We note that the process is adapted to and it satisfies the following assumption.

Assumption 5

For al , a.s..

Moreover, the stochastic error has a vanishing second moment that depends on the increasing number of samples

taken at each iteration.

Assumption 6

There exist such that, for all ,

(15)

For all and , the stochastic error is such that

(16)

The bound for the stochastic error in (16) can be obtained as a consequence of some milder assumptions; we refer to [13, Lem. 4.2], [14, Lem. 3.12], [15, Lem. 6] for more details. Concerning the batch size, the law in (15) is standard in the sample average approximation literature [14, Eq. 11], [17, Eq. v-SPRG].

Furthermore, since the preconditioning matrix must be positive definite, we postulate the following assumption on the parameters.

Assumption 7

Let be the cocoercivity constant as in Lemma 2, and the step sizes , and satisfy, for all ,

(17)

where indicates the entry of the matrix .

For example, we can obtain conditions (17) imposing that the preconditioning matrix to be diagonally dominant which, since it is symmetric, implies that is positive definite [8].

We are now ready to state our convergence result.

Theorem 1

Let Assumptions 1-7 hold. Then, the sequence generated by Algorithm 1 with as in (14) converges a.s. to a v-SGNE of the game in (3).

The iterations of Algorithm 1 are obtained by expanding (12) and solving for , and . Therefore, Algorithm 1 is a SPRG iteration as in (12). The convergence of the sequence to a v-GNE of the game in (3) then follows by [17, Prop. 10] and Lemma 1 since is cocoercive by Lemma 2.

Remark 3

We note that adopting a SA scheme is not possible in this case because a vanishing step should be taken to control the stochastic error [11]. However, having a time-varying step implies using a variable metric, induced by the preconditioning matrix which depends on , and , for the convergence analysis. Although analysing a variable metric is possible, the matrix should satisfy additional assumptions that typically do not hold if the step size is vanishing [23, Prop. 3.4].

V Technical discussion on weak sharpness and cocoercivity

The original proof of the SPRG presented in [17] for SVI shows convergence under the assumption of monotonicity and weak sharpness. The weak sharpness property was first introduced to characterize the minima of

(18)

with [24]. It was presented as en extension of the concept of strong (or sharp) solution, i.e., for all

which holds if there is only one minimum. For generalizing non-unique solutions, the following definition was proposed in [24]: a set is a set of weak sharp minima for the function if, for all and ,

(19)

where . We note that a strong solution is also a weak sharp minimum while the contrary holds only if the solution is unique [24].

The concept was later extended to variational inequalities in [25], using the formal definition

(20)

which was already proved to be equivalent to (19) for the problem in when .

Unfortunately, the characterization in (20) is hard to use in a convergence proof. Therefore, more practical conditions have been proposed. The first one [25] relies on the gap function and it reads as

(21)

Another condition, used in the convergence proof of the SPRG [17], was proposed in [12]:

(22)

for all and . For the weak sharpness definition in (20) to be equivalent to (21) and (22), the (pseudogradient) mapping should have the F-unique property, i.e., should be at most a singleton [18, Section 2.3.1]. The class of operators that certainly have this property is that of monotone operators, namely, a monotone mapping such that for all

If a mapping is monotone, then (20) is equivalent to (21) and (22) [25, Thm. 4.1],[26, Prop. 2].

The monotone property does not necessarily hold for the extended operator in (10), even if it holds for . However, it holds if the operator is cocoercive [18, Def. 2.3.9]. These considerations motivate our assumption. For more details on monotone operators and the weak sharpness property, we refer to [27, 28, 25, 29].

We conclude this section with some examples showing that the condition in (22) may hold also if the mapping is not monotone and that the domains are relevant for the validity of the assumption.

Example 1

[25] Consider the variational inequality in (5) where and . The mapping is pseudomonotone but not monotone on . The solution set is and it holds that

(23)

Therefore, satisfies (21) with but, for any , and . Thus, the solution set is not weakly sharp.

Example 2

Consider the mapping and the associate variational inequality in (5) with . Then the mapping is monotone but not monotone on . The solution set is and, similarly to (23), the conditions (21) and (22) hold.

Now, let . In this case, there is only one solution and . However, (22) reads as

which is false.

Vi Convergence under uniqueness of solution

In light of the considerations in Section V, we know that a unique solution is also a weak solution and that (22) may hold even if the mapping is not monotone. Therefore, here we consider the case of merely monotone operators but with unique solution and prove that the proposed (non-preconditioned) Algorithm 2 converges to a v-SGNE.

Initialization: and
Iteration : Agent
(1): Updates

(2): Receives for all and for , then updates:

Algorithm 2 Stochastic Relaxed Forward Backward (SRFB)

Unlike Algorithm 1, in Algorithm 2 also should be updated, but only one communication round is required.

To obtain the iterates in Algorithm 2, a different splitting should be considered. Specifically, let

(24)

Lemma 1 guarantees that the zeros of are the same as the zeros of in (8).

Since the distribution of the random variable is unknown, we replace with

(25)

where is an approximation of the expected value mapping in (4) given some realizations of the random vector .

Then, Algorithm 2 in compact form reads similarly to Algorithm 1:

(26)

where contains the inverse of step size sequences

(27)

and , , are diagonal matrices. We note that is not a preconditioning matrix in this case.

Now, to ensure that and have the properties that we use for the convergence result, we make the following assumption.

Assumption 8

as in (4) is monotone and -Lipschitz continuous for some .

Then, the two operators and in (24) have the following properties.

Lemma 3

Let Assumptions 1 and 8 hold and let . Then, and in (24) have the following properties.

  1. is monotone and -Lipschitz continuous.

  2. The operator is maximally monotone.

  3. is monotone and -Lipschitz continuous.

  4. is maximally monotone.

it follows from [8] and [15].

To guarantee that the weak sharpness property holds, we assume to have a strong solution.

Assumption 9

The SVI in (5) has a unique solution.

We can now state the convergence result.

Theorem 2

Let Assumptions 1-3 and 5-8 hold. Then, the sequence generated by Algorithm 2 with as in (14) converges a.s. to a v-SGNE of the game in (3).

The iterations of Algorithm 2 are obtained by expanding (26) and solving for , and . Therefore, Algorithm 2 is a SPRG iteration as in (26). The convergence of the sequence to a v-GNE of the game in (3) then follows by [17, Prop. 10] and Lemma 1 since is monotone by Lemma 3 and has a unique solution.

Vii Numerical Simulations

Let us propose some numerical evaluations to validate the analysis: an illustrative example and a Nash-Cournot game. While the first comes from Example 2, the second is a realistic application to an electricity market with capacity constraints [8, 17].

All the simulations are performed on Matlab R2019b with a 2,3 GHz Intel Core i5 and 8 GB LPDDR3 RAM.

Vii-a Illustrative example

We start with the stochastic counterpart of Example 2, that is, a monotone (non-cocoercive) stochastic Nash equilibrium problem with two players with strategies and respectively, and pseudogradient mapping

The random variables are sampled from a normal distribution with mean 1 and finite variance, following Assumption

6. The problem is unconstrained and the optimal solution is . The step sizes are taken to be the highest possible and we compare our SpPRG with the stochastic distributed preconditioned forward–backward (SpFB) which is guaranteed to converge under the same cocoercivity assumption with the SAA scheme [15].

Figure 1 shows that the SpFB does not converge while, due to the uniqueness of the solution, the SpPRG does.

Fig. 1: Relative distance of the primal variable from the solution.

Vii-B Nash-Cournot game with market capacity constraints

Now, we consider an electricity market problem that can be casted as a network Cournot game with markets capacity constraints [8, 3, 17]. We consider a set of companies selling their product to a set of markets. Each generator decides the quantity of energy to deliver to the markets it is connected with. Each company has a local constraint, i.e., a production limit, of the form where each component of is randomly drawn from . Each company has a cost of production , where is a given constant, that is not uncertain. For simplicity, we assume the transportation costs are zero.

Each market has a bounded capacity , randomly drawn from . The collective constraints are then given by where and each specifies in which market each company participates.

The prices of the markets are collected in . The uncertainty variable, which represents the demand uncertainty, appears in this functional. is supposed to be a linear function and reads as . Each component of is taken with a normal distribution with mean and finite variance. The entries of are randomly taken in .

The cost function of each agent is then given by

and it is influenced by the variables of the companies selling in the same market as in [8, Fig. 1]. The dual variables graph is a cycle graph with the addiction of the edges and [8].

We simulate the SpFB, the forward-backward-forward (SFBF) and the extragradient (SEG) algorithms to make a comparison with our SPRG and SpPRG, using the SAA scheme. The parameters , and are taken to be the highest possible that guarantee convergence.

As a measure of the distance from the solution, we consider the residual, , which is equal zero if and only if is a solution. The plots in Fig. 2 shows how the residual varies in the number of iterations while the plot in Fig. 3 shows the number of times that the pseudogradient mapping is computed. As one can see from the plots, the performances of SpPRG and SPRG are very similar. The difference in the trajectory is related to the different step sizes which depend on the Lipschitz constant of in (10) and in (24) respectively.

Fig. 2: Relative distance of the primal variable from the solution.
Fig. 3: Relative distance of the primal variable from the solution.