Distributed GNE seeking under partial-decision information over networks via a doubly-augmented operator splitting approach

08/13/2018 ∙ by Lacra Pavel, et al. ∙ 0

We consider distributed computation of generalized Nash equilibrium (GNE) over networks, in games with shared coupling constraints. Existing methods require that each player has full access to opponents' decisions. In this paper, we assume that players have only partial-decision information, and can communicate with their neighbours over an arbitrary undirected graph. We recast the problem as that of finding a zero of a sum of monotone operators through primal-dual analysis. To distribute the problem, we doubly augment variables, so that each player has local decision estimates and local copies of Lagrangian multipliers. We introduce a single-layer algorithm, fully distributed with respect to both primal and dual variables. We show its convergence to a variational GNE with fixed step-sizes, by reformulating it as a forward-backward iteration for a pair of doubly-augmented monotone operators.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Generalized Nash equilibrium (GNE) problems in games with shared coupling constraints arise in various network scenarios where a set of players (agents) compete for limited network resources, e.g. power grids and smart grids, [2], optical networks, [3], wireless communication networks, [4, 5], electric vehicle charging, [7]. The study of GNE dates back to [8, 9]; a historical review is provided in [10], [11]. Distributed GNE computation in monotone games has seen an increasing interest in recent years, [5, 6, 19, 7, 12, 13, 14, 15, 16, 17, 18]. Most works assume that each player has access to all other agents’ decisions - the classical setting of full-decision information, either by observation or by a central node coordinator.

There are many current networked applications where agents may only access or observe the decisions of their neighbours, and there is no central node to provide them with global information, i.e., a partial-decision information setting. The assumption of information exchange is motivated in networks where there is no central node that has bidirectional communications with all players to provide them with global information, as in peer-to-peer networks. Application scenarios range from spectrum access in cognitive radio networks, where users adaptively adjust their operating parameters based on interactions with the environment and other users in the network, [20], congestion games in ad-hoc networks, [21], to networked Nash-Cournot competition, [22], and opinion dynamics in social networks, [23], [24]. These examples are non-cooperative in the way decisions are made (each agent minimizes its own cost function), while agents exchange locally information with neighbours to compensate for the lack of global information on others’ decisions. The first results on distributed NE seeking under such partial-decision information have been for finite-action games, [25], and for aggregative games with no coupling constraints, [26]. Results were extended to general continuous-kernel games in [27, 31, 28, 29, 30], for NE seeking problems only, in games with no coupling constraints. Inspired by work on NE seeking under partial-decision information, [26], and by the recent elegant, operator-theoretic approach to GNE problems, [16], [17], in this paper we consider GNE seeking in games with affine coupling constraints, under partial-decision and local information exchange over an arbitrary network.

Literature review: Distributed (variational) GNE computation is an active research area, but existing results are for the classical setting of full-decision information. Initial results were developed based on a variational inequality (VI) approach, [10], [5]. For (pseudo)-monotone games, [5] adopts a single-layer Tikhonov regularization primal-dual algorithm, [6] proposes a primal-dual gradient approach, while [19] proposes a payoff-based algorithm, all with diminishing step-sizes. Recently, an operator-splitting approach has proved to be very powerful; it allows the design of GNE algorithms that are guaranteed to globally converge with fixed step-sizes, with concise convergence proofs. Most results are for aggregative games, [7], [13, 14, 15, 16]. In [14, 15, 16], algorithms are semi-decentralized, requiring a central node (coordinator) to broadcast the common multipliers and/or aggregative variables, hence a star topology. This is relaxed in [13] by combining a continuous-time consensus dynamics and a projected gradient, still for aggregative games. For games with generally coupled costs and affine coupling constraints, distributed and center-free GNE seeking is investigated via an operator approach in [17, 18, 36]: a forward-backward algorithm, convergent in strongly monotone games [17, 36], and preconditioned proximal algorithms for monotone games [18]. Players communicate the local multipliers over a network with arbitrary topology, in a distributed, peer-to-peer manner, but each agent has access to the decisions of all other agents that influence his cost, hence full-decision information.

Contributions: Motivated by the above, in this paper we consider distributed GNE seeking in a partial-decision information setting via an operator-splitting approach. We propose a fully distributed GNE seeking algorithm for games with generally coupled costs and affine coupling constraints, over networks with an arbitrary topology. To the best of our knowledge, this is the first such algorithm in the literature. Based on a primal-dual analysis of the variational inequality KKT conditions, we reformulate the problem as that of finding zeros of a sum of monotone operators and use the Laplacian matrix to distribute the computations. Different from [17] (perfect opponents’ decision information), herein we distribute both the primal and the dual variables. To account for partial-decision information, we endow each agent with an auxiliary variable that estimates the other agents’ decisions (primal variables), as in NE seeking over networks, [31, 28]. Compared to [17, 36], this introduces technical challenges, as a change in an estimate induces a nonlinear change in an agent’s dynamics. We make use of two selection matrices and we incorporate the Laplacian in an appropriate manner to do double duty, namely to enforce consensus of the local decision estimates (primal variables) and of the local multipliers (dual variables). Compared to [1], here we relax the assumption of cocoercivity of the extended pseudo-gradient. Under Lipschitz continuity of the extended pseudo-gradient, we prove convergence with fixed step-sizes over any connected graph, by leveraging monotone operator-splitting techniques, [34]. Specifically, we reformulate the algorithm as a forward-backward iteration for doubly-augmented monotone operators, and distribute the resolvent operation via a doubly-augmented metric matrix.

The paper is organized as follows. Section II gives the notations and preliminary background. Section III formulates the game. Section IV introduces the distributed GNE seeking algorithm and reformulates it as an operator-splitting iteration. The convergence analysis is presented in Section V, numerical simulations in Section VI and concluding remarks are given in Section VII. Some of the proofs are placed in the appendix.

Ii Preliminary background

Notations.

For a vector

, denotes its transpose and the norm induced by inner product . For a symmetric positive-definite matrix , , and

denote its minimum and maximum eigenvalues. The

-induced inner product is and the -induced norm, . For a matrix , let denote the 2-induced matrix norm, where

is its maximum singular value. Let

and . For , or denotes the stacked vector obtained from vectors , the block diagonal matrix with on the main diagonal. and are the null and range space of matrix , respectively, while stands for its entry.

denotes the identity matrix in

. Denote or as the Cartesian product of the sets .

Ii-a Monotone operators

The following are from [34]. Let be a set-valued operator. The domain of is where is the empty set, and the range of is . The graph of is ; the inverse of is defined through its graph as . The zero set of is . is called monotone if , It is maximally monotone if is not strictly contained in the graph of any other monotone operator. The resolvent of is , where is the identity operator. is single-valued and if is maximally monotone. The composition of and is denoted by . The sum is defined as . If and are maximally monotone operators and , then is also maximally monotone. If is single-valued, then , [34, Prop. 25.1], where denotes the set of fixed points of .

For a proper lower semi-continuous convex (l.s.c.) function , its sub-differential is is a maximally monotone operator. , is the proximal operator of . Define the indicator function of as if and if For a closed convex set , is a proper l.s.c. function and is the normal cone operator of , .

An operator is nonexpansive if it is Lipschitz, i.e., . is averaged (), if there exists a nonexpansive operator such that . By [34, Prop. 4.25], given , , where denotes the class of averaged operators, if and only if :
(i): .
(ii): .
If , is also called firmly nonexpansive. If is maximally monotone, is firmly nonexpasive, [34, Prop. 23.7]. Let the projection of onto be , with . If is closed and convex, is firmly nonexpansive since is maximally monotone [34, Prop. 4.8]. is called cocoercive if , for , i.e., . If is convex differentiable, with Lipschitz gradient , then is cocoercive (cf. Baillon-Haddad theorem, [34, Thm. 18.15]).

Ii-B Graph theory

The following are from [35]. Let graph describe the information exchange among a set of agents, where is the edge set. If agent can get information from agent , then and agent belongs to agent ’s neighbour set , . is undirected when if and only if . is connected if any two agents are connected. Let be the weighted adjacency matrix, with if and otherwise, and , where . Assume . The weighted Laplacian of is When is connected and undirected, 0 is a simple eigenvalue of , , ; all other eigenvalues are positive. Let the eigenvalues of in ascending order be , , where is the maximal weighted degree.

Iii Game formulation

Consider a group of agents (players) , where each player controls its local decision (strategy or decision) . Denote as the decision profile, i.e., the stacked vector of all the agents’ decisions where . We also write as where denotes the decision profile of all agents’ decisions except player . Agent aims to optimize its objective function , coupled to other players’ decisions, with respect to its own decision over its feasible decision set. Let the globally shared, affine coupled constrained set be

(1)

where is a private feasible set of player , and , its local data. Let . A jointly-convex game with coupled constraints is represented by the set of inter-dependent optimization problems

(2)

where is the feasible decision set of agent . A generalized Nash equilibrium (GNE) of game (2), (1) is a profile at the intersection of all best-response sets,

(3)
Assumption 1

For each player , is continuously differentiable and convex in , given , and is non-empty compact and convex. is non-empty and satisfies Slater’s constraint qualification.

Denote and . Suppose is a GNE of game (2), (1) then for agent , is the optimal solution to the following convex optimization problem:

(4)

A primal-dual characterization can be obtained via a Lagrangian for each agent ,

(5)

with dual variable (multiplier) . When is an optimal solution to (4), there exists such that the following KKT conditions are satisfied:

(6)

Equivalently, using the normal cone operator,

(7)

Denote . By [10, Thm. 8, §4] when satisfies KKT conditions (7), is a GNE of game (2), (1).

A GNE with the same Lagrangian multipliers for all the agents is called variational GNE, [10], which has the economic interpretation of no price discrimination, [32]. A variational GNE of game (2), (1) is defined as solution of the following :

(8)

where is the pseudo-gradient of the game defined as:

(9)

solves if and only if there exists a such that the KKT conditions are satisfied, [33, §10.1],

(10)

where , or component-wise,

Assumption 1 guarantees existence of a solution to (8), by [33, Cor. 2.2.5]. By [10, Thm. 9, §4], every solution of (8) is a GNE of game (2). Furthermore, if together with satisfies the KKT conditions (10) for (8), then satisfies the KKT conditions (7) with , hence is variational GNE of game (2).

Our aim is to design an iterative algorithm that finds a variational GNE under partial-decision information over a network with arbitrary topology , by using an operator-theoretic approach. We first review typical iterative algorithms under full-decision information, where each agent has access to the others’ decisions.

Iii-a Iterative Algorithm under Full-Decision Information

Assumption 2

is strongly monotone and Lipschitz continuous: there exists and such that for any pair of points and , and .

Strong monotonicity of is a standard assumption under which convergence of projected-gradient type algorithms is guaranteed with fixed step-sizes, e.g. [6],[7], [17], [16]. Under Assumption 1, 2, the , (8), has a unique solution (cf. [33, Thm. 2.3.3]), thus the game (2) has a unique variational GNE. Assuming each player has access to the others’ decisions , i.e., full-decision information, a primal-dual projected-gradient GNE algorithm is

(11)

where , denote , at iteration and and are fixed step-sizes. The dual variable is handled by a center (coordinator) as in [16] hence (11) is semi-decentralized.

Algorithm (11) is an instance of an operator-splitting method for finding zeros of a sum of monotone operators, [34, §25]. To see this, note that the KKT conditions (10) can be written as where the operator is defined by the concatenated right-hand side of (10). can be split as , where operators , are defined as

(12)

Algorithm (11) can be obtained as a forward-backward iteration, [34, §25.3], for zeros of , where is a metric matrix. We note that different GNE seeking algorithms can be obtained for different splitting of , with convergence conditions dependent on monotonicity properties of and . Notice that (III-A) is maximally monotone (similar arguments for this can be found in Lemma 4), and under Assumption 2, is cocoercive. Convergence of (11) to , can be proved for sufficient conditions on the fixed-step sizes such that .

Iv Distributed Algorithm under Partial-Decision Information

In this section we consider a partial-decision information setting, where the agents do not have full information on the others’ decisions . We propose an algorithm that allows agents to find a variational GNE based on local information exchange with neighbours, over a communication graph with arbitrary topology, under the following assumption.

Assumption 3

is undirected and connected.

Our approach is based on the interpretation of the KKT conditions (10) as a zero-finding problem of a sum of operators. To deal with (incomplete) partial-decision information and to distribute the computations, we introduce estimates and lift the original problem to a higher-dimensional space. This space (called the augmented space), is doubly-augmented (in both primal and dual variables), and the original space is its consensus subspace. We appropriately define a pair of doubly-augmented operators, such that any zero of their sum lies on the consensus subspace, and has variational GNE and as its components.

We describe next the algorithm variables. Agent controls its local decision , and a local copy of multiplier (dual variable) for the estimation of in (10). To cope with partial-decision information, we endow each player with an auxiliary variable that provides an estimate of other agents’ primal variables (decisions), as done in [31, 28] for NE seeking. Thus agent maintains , where is player ’s estimate of player ’s decision and is its decision. Note that , where represents player ’s estimate vector without its own decision . In steady-state all estimates should be equal, i.e., and . Each agent uses the relative feedback from its neighbours such that in steady-state these estimates, on both primal and dual variables, agree one with another. An additional local auxiliary variable is used for the coordination needed to satisfy the coupling constraint and to reach consensus of the local multipliers (dual variables) . Agents exchange local via the arbitrary topology communication graph . if player can receive from player , where denotes its set of neighbours.

The distributed algorithm for player is given as follows.

Algorithm 1

  Initialize: , , , .
Iteration:

  Here , , , denote , , , at iteration , is a design parameter, are fixed step-sizes of player , and the weighted adjacency matrix of .

Remark 1

The update for in Algorithm 1 employs a projected-gradient descent of the local Lagrangian function in (5) with an extra proportional term of the consensual errors (disagreement) between his primal variables and his neighbours’ estimates. The updates for and can be regarded as discrete-time integrations for the consensual errors of local decision estimates and dual variables. Finally, is updated by a combination of the projected-gradient ascent of local Lagrangian (5) and a proportional-integral term for consensual errors. Each player knows only its local data in game (2), , , and , own private information, i.e., cost function, preference and decision ability. characterizes how agent is involved in the coupled constraint (shares the global resource), assumed to be privately known by player . The globally shared constraint couples the agents’ feasible decision sets, but is not known by any agent.

Remark 2

Compared to algorithm (11), Algorithm 1 is completely distributed (without any central coordinator), i.e., primal-distributed and dual-distributed over . The algorithms in [17], [31] are special cases of Algorithm 1. When each agent has access to all players’ decisions that affect its cost, the estimates are not needed (set ), and Algorithm 1 reduces to the dual-distributed, perfect-information case one in [17] (dual distributed). On the other hand, in a game with no coupling constraints (set , ), the (hence the ) are not needed, and Algorithm 1 reduces to a discrete-time version of the primal-distributed dynamics in [31].

Next, we write Algorithm 1 in compact form, using two matrices to manipulate the selection of agent ’s decision variables, , and estimate variables, . Let

(13)
(14)

where , . Hence selects the -th -dimensional component from an -dimensional vector, while removes it. Thus, and . With , the stacked decisions can be written as , where and . Similarly, the stacked estimates are , where . These two matrices, and , play a key role in the following. Using (13), it can be seen that both are full row rank and moreover,

(15)

Furthermore, with , we can write . With these notations, we write Algorithm 1 in stacked form, using boldface notation for stacked variables (all local copies).

Lemma 1

Let , , , . Then, Algorithm 1 is equivalently written in stacked notation as

(16)
(17)
(18)
(19)

where is defined as

(20)

, , , , , , , , , .

Remark 3

In Algorithm 1, instead of evaluating its gradient at actual decisions, as in , each player evaluates its gradient at local estimates, . The stacked form , (20), called the extended pseudo-gradient, is the extension of , (9) to the augmented space of decisions and estimates. When these estimates are identical, for all , then .

Based on Lemma 1, we show next that Algorithm 1 can be written as a forward-backward iteration for finding zeros of the sum of two doubly-augmented operators and , where , are related to , , (III-A), and is a (preconditioning) metric matrix. Let , where . Define , as

(21)

where , , .

Let the matrix be defined as

(22)

where , , and are similarly defined from .

Lemma 2

Let , , , as in (IV), (22). Suppose that and is maximally monotone. Then the following hold:
(i): Algorithm 1 is equivalent to

(23)
(24)

where and .
(ii): Any limit point of Algorithm 1 is a zero of and a fixed point of .

Remark 4

Algorithm 1, written as (24) is a forward-backward iteration for finding zeros of , or fixed-point iteration for [34, §25.3]. It alternates a forward step , and a backward step