Distributed payoff allocation in coalitional games via time varying paracontractions

11/28/2019 ∙ by Aitazaz Ali Raja, et al. ∙ 0

We present a partial operator-theoretic characterization of approachability principle and based on this characterization, we interpret a particular distributed payoff allocation algorithm to be a sequence of time-varying paracontractions. Further, we also propose a distributed algorithm, under the context of coalitional game, on time-varying communication networks. The state in the proposed algorithm converges to a consensus within, the predefined, desired set. For convergence analysis, we rely on the operator-theoretic property of paracontraction.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Coalitional game theory provides an analytical framework and mathematical formalism, to study the behavior of selfish and rational agents, when they are willing to cooperate. Interestingly, this scenario arises in many applications, such as demand side energy management [Han et al. (2018)], in power networks for transmission cost allocation [Zolezzi and Rudnick (2002)] and cooperation between microgrids in distribution networks [Saad et al. (2011)], in various areas of communication networks by [Saad et al. (2009)], [Saad et al. (2008)] and as conceptual foundation for coalitional control [Fele et al. (2017)].
Specifically, a coalitional game with transferable utility

consists of a set of agents referred as players, who can form coalitions, and a characteristic function that determines the

value of each coalition. Note that a selfish agent will cooperate with other agents only if this coalition results in increasing its own benefit. The latter is determined by the payoff the agent receives from the value generated by a coalition. The design of criteria for determining this payoff has received acute attention by research community, such as Scarf (1967), Shapley (1953), Schmeidler (1969), Maschler et al. (1971). The solutions proposed determine the stability of a coalition, i.e., whether the coalition remains intact or gets defected by its agents. One of the most widely studied solution concepts is the CORE which ensures the stability of a game.

The problem we address in this paper is finding a payoff that belongs to CORE and hence encourages cooperation. Our practical treatment of this problem is in a multi-agent scenario, where players interact autonomously and in distributed manner to arrive at common agreement on a payoff vector in the CORE. In this direction,

Lehrer (2003) presented an allocation process which converges to the CORE (or if this is empty, to a least-CORE). Smyrnakis et al. (2019) also consider an allocation process but under noisy observations and dynamic environment. Bauso et al. (2014) provide conditions for an averaging process, with dynamics subject to controls and adversarial disturbances, under which the allocations converge to consensus in the desired set. Nedich and Bauso (2013) propose an elegant distributed bargaining algorithm which converges to a random CORE payoff allocation. The key inspiration, however, of our work is the distributed payoff allocation algorithm proposed by Bauso and Notarstefano (2015). Their algorithm is based on the approachability principle, which is a geometric condition introduced in Blackwell’s approachability Theorem [Blackwell (1954)]. The approachability principle provides a way to approach a particular set and hence can be exploited to reach the CORE in the context of coalitional game theory.

Contribution: In this paper, we first show that the approachability condition contains a paracontraction operator. Briefly, an operator is said to be a paracontraction if, for any fixed point and , where , it holds that . These operators form the subclass of, perhaps more known, quasi-non-expansive mappings [Ernest Ryu and Boyd (2016)].
Secondly, we propose a distributed payoff allocation algorithm, in context of coalitional games over time-varying communication networks. The state of proposed algorithm converges to a consensus vector that belongs to the CORE. Our approach to prove convergence of our algorithm relies on the paracontraction property of the adopted operator.

Organization of the paper: In Section 2, we provide the mathematical background for coalitional games and distributed allocation process. In Section 3, we discuss the approachability principle and recall the distributed payoff allocation algorithm by Bauso and Notarstefano (2015). In Section 4, we provide a partial operator-theoretic characterization of the approachability principle, and we discuss algorithm in [Bauso and Notarstefano (2015)]. In Section 5, we propose an algorithm for distributed allocation in coalitional games and establish its convergence using operator-theoretic properties. Further, we asses the convergence speed of proposed algorithm in Section 6, and in Section 7, we conclude the paper.

Notation: and denote the set of real and natural numbers, respectively. Given a mapping denote the set of fixed points. Id denotes the identity operator. For a closed set the mapping : denotes the projection onto i.e., denotes the Kronecker product between the matrices and

denotes an identity matrix of dimension

. denotes the distance of from a closed set i.e., .

2 Mathematical Background on coalitional games

A coalitional game consists of a set of agents, indexed by , who cooperate to achieve selfish interests. This cooperation results in generation of utility as defined by the characteristic function . Formally, (Coalitional game): A transferable utility (TU) coalitional game is a pair , where is the index set of the agents and is a characteristic function which assigns a real value, , to each coalition . is the value of so-called grand coalition. By convention, . The idea of coalitional game is that the value attained by a coalition , i.e. , has to be distributed among the members of the coalition, thus each agent receives a certain payoff. (Payoff vector): Let be a coalition of coalitional game . A payoff vector is a vector . Where represents the share of agent of .

Let us state two important characteristics of a payoff vector which will further help us in explaining the solution concept of a coalitional game. First, for a game with a grand coalition , a payoff vector is said to be efficient if . In words, all of the value generated by grand coalition will be distributed among the agents. Second, a payoff vector is rational if for every possible coalition we have . Note that this should also hold for singleton coalitions i.e. . It means that, payoff allocated to each agent should be at least equal to what they can get individually or by forming any coalition other than .
A payoff vector which is both efficient and rational lies in the CORE. CORE is the solution concept that relates with the stability of a grand coalition. Where, the idea of stability, in this context, is based on the disinterest of agents in defecting a grand coalition. Formally, (CORE): The CORE of a coalitional game () is the following set of payoff vectors:

(1)

Each payoff allocation that belongs to CORE stabilizes the grand coalition. It implies that no agent or coalition has an incentive to defect from the grand coalition.
In the sequel, we deal with the grand coalition only, therefore we use the CORE as the solution concept. Note from (1) that is closed and convex. We also assume the CORE to be non-empty through out the paper. Next, we discuss a possible strategy of finding the payoff vector, in a coalitional game , that belongs to CORE, in (1). Centralized methods for finding a vector do not capture realistic scenarios of interaction among autonomous selfish agents. Thus, distributed methods are employed that allow agents to autonomously reach a common agreement on a payoff allocation, .
Generally, the distributed allocation is an iterative procedure in which, at each step, an agent proposes a utility distribution by averaging the proposals of all agents and introducing an innovation factor. This procedure aspires to finally reach at a mutually agreed payoff among participating agents. Eventually the proposed utility distributions must reach consensus. (Consensus set): The consensus set is defined as:

(2)


Therefore, in this paper, we consider the problem of computing a mutually agreed, payoff allocation vector in the CORE, i.e., , via a iterative distributed allocation, i.e.,

3 Approachability principle and Distributed payoff allocation

3.1 Approachability principle

We now discuss a geometric principle which can guarantee the convergence of a payoff allocation sequence to a target set, which in our coalitional game theory context, is the CORE , as in (1).This principle, which we refer to as approachability principle, is the geometric concept behind celebrated approachability theorem by Blackwell presented in [Blackwell (1954)].

Figure 1: Geometric interpretation of the approachability principle.

(Approachability Principle)[Lehrer (2003), 3.2], Let be a sequence of uniformly bounded vectors in with running average , and let be a non-empty, closed and convex set. If the sequence satisfies the condition,

(3)

then In Figure 1, we illustrate the approachability condition in (3

). Let us give a geometric interpretation: the hyperplane through the point

, perpendicular to the vector , which is the first term in (3), separates the space into the half-spaces . The the approachability condition requires that, the innovation and the average should not lie in the same half-space.
Among others, Bauso and Notarstefano (2015) have used the approachability principle to design a distributed payoff allocation algorithm which converges to a consensus vector in the CORE in (1). Let us recall their setup and solution algorithm in next subsection.

3.2 A time-varying Distributed payoff allocation process

Consider a set of agents who synchronously propose a distribution of utility at each discrete time step . Specifically, each agent proposes a payoff distribution , where the th element denotes the share of agent proposed by agent . Then, each agent computes by averaging the proposals by his neighboring agents and then by generating an innovation vector as follows:

(4)

where is a positive sequence of step sizes, with , and represents an adjacency matrix.
Now, Let the communication graph vary over time as . Specifically, means that there is an active link between agents and at time . In [Bauso and Notarstefano (2015), Assumption 2], the graph sequence is assumed to be connected. There exists an integer such that the graph is strongly connected, for all . The communication links in are weighted using an adjacency matrix , whose element represents the weight assigned by agent to the payoff distribution proposed by agent , . By [Bauso and Notarstefano (2015), Assumption 1], the adjacency matrix is always doubly stochastic with positive diagonal. For all , the matrix satisfies following conditions:

  1. It is doubly stochastic;

  2. its diagonal elements are strictly positive, i.e., ;

  3. such that whenever .

Furthermore, at each time , the agents generate an innovation vector in (4), satisfying approachability condition, as formulated in (3). Specifically, let , with as in (4), then following is postulated in [Bauso and Notarstefano (2015), Assumption 4]: For each , the innovation vector in (4) satisfies the following inequality:

(5)

where is the CORE set as in (1). Moreover, to fulfil the conditions of the approachability principle, the innovation vector is uniformly bounded, [Bauso and Notarstefano (2015), Assumption 4]. Let be innovation vector in (4). There exist , such that . The main result regarding the iteration in (4) by Bauso and Notarstefano (2015) is that, if Assumptions 3.23.2 hold then the average allocation vector will converge to the set . In the context of coalitional game theory, this implies that through the distributed allocation process in (4), the agents will reach a common agreement on the payoff distribution, which lies in the CORE.

4 operator theoretic characterization

4.1 Approachability principle as a paracontraction

In this subsection, we aim at providing an operator-theoretic characterization of the approachability condition in (5), and present an interesting operator contained by approachability condition which holds a paracontraction property. To show that, we first define the notion of paracontraction. (Paracontraction): A continuous mapping is a paracontraction, with respect to a norm on , if

for all . The approachability condition in (5), given provides us the criterion for generating an innovation vector to be used in the iterative process in (4

). In the next statement, we will present an alternative formulation for the approachability condition which, interestingly, is the sum of a paracontracting operator and arbitrary vectors with specific geometric meaning.

Let , be the over-projection operator, be an arbitrary vector that belongs to the hyperplane orthogonal to the vector in (5) and be a vector orthogonal to in the direction opposite to vector , (Figure 2). Then, the following equation corresponds exactly to the approachability condition in (5):

(6)
(a)
(b)
Figure 2: Illustration of the approachability condition as in Equation (6): projection and over-projection (a); innovation (b).

In Figure 2, we geometrically illustrate Equation (6) for some . To show that (6) corresponds to the approachability condition, let us plug (6) into (5). In the remainder of the proof, we drop the dependence on for ease of notation.


Since all the steps are equivalent and the vectors and can be chosen arbitrarily for each given , and since any point in can be written in the form in (6), we conclude that (6) is equivalent to the approachability condition in (5). Let us now consider the particular case of (6) with , and define the dependence of from via an operator :

(7)

The operator in (7) is a mapping from to which, by Lemma 4.1, satisfies the approachability condition in (5). Using this operator , we can give the following representation to the process of generation of an innovation vector in (4), which is equivalent to the particular case in (7) of the approachability condition.

(8)

Next, we present an operator-theoretic property of the operator in the following statement. The operator defined in (7)(8) is a paracontraction. Before presenting the proof of Theorem 4.1, we provide two technical statements, which we exploit later in the proof. (Projection and Over-projection operators): Let be a non-empty, closed and convex set. Then, with respect to the Euclidean norm :

  1. the projection operator is a paracontraction;

  2. the over projection operator, , is non-expansive.

(i): If is closed and convex then is a paracontraction, [Elsner et al. (1992), Example 2].
(ii): By [Ernest Ryu and Boyd (2016), Subsection 3.1]. Let be a paracontraction, be a non-expansive operator, with and . Then, is a paracontraction. Let and . Then:

where we have used the triangular inequality and then the definition of paracontraction for . Therefore, with , we obtain the definition of paracontraction. Lemma 4.1 also holds if both operators are paracontractions (with the same proof). Given these results, we are now ready to present the proof of Theorem 4.1.

(Theorem 4.1): At each time an agent generates an innovation vector in (4), satisfying the restricted approachability condition in (7). By Lemma 4.1, the operator in (8) is a convex combination of a paracontraction, and a non-expansive operator, . Thus, by Lemma 4.1, it is a paracontraction.

4.2 Distributed allocation process as a sequence of time varying paracontractions

The result in Theorem 4.1 further allows us to characterize an operator-theoretic property of the iteration in (4). We show that, under a particular case of approachability condition in (7), the iteration generates a sequence of time varying paracontractions. To prove this, we recall two useful results related to paracontractions. (Composition of paracontracting operators): Suppose are paracontractions with respect to same norm and . Then the composition is a paracontraction with respect to the norm and , [Fullmer and Morse (2018), Prop. 1].

(Doubly stochastic matrix): Let

be a doubly stochastic matrix with strictly positive diagonal elements. Then, the linear operator defined by the matrix is a paracontraction with respect to the mixed vector norm , [Fullmer and Morse (2018), Prop. 5]. Using the operator in (8) and as in (5), we can rewrite (4) as:

(9)

Note that, the step-size sequence in (4) is specified to be by Bauso and Notarstefano (2015). Here, we can generalize it subject to the following assumption. Let be a sequence such that , , and .

Let us also define an operator , which in turn allows us to represent the iteration in (9) more concisely as:

(10)

With the latter formulation, we can now conveniently characterize the paracontraction property of the operator , according to the corollary below. Let the operator be as in (8). Then, for each , the operator in (10) is a paracontraction. By Theorem 4.1, the operator is a paracontraction. Furthermore, by Proposition 4.2 and 4.2, the composition is also a paracontraction. This fact and Proposition 4.2 imply that for each the operator is a convex combination of paracontractions and hence, by Remark 4.1 on Lemma 4.1, is a paracontraction. Corollary 4.2 also holds if, for all in (9). The results in Theorem 4.1 and Corollary 4.2 provide an interesting operator-theoretic insight into the structure of algorithm presented by Bauso and Notarstefano (2015). We use this insight to design our own distributed payoff allocation algorithm, which we present in the next section along with its convergence proof.

5 Distributed allocation via paracontraction operators over time-varying networks

In this section, we present our distributed allocation algorithm and exploit the results derived in Section 4 to prove its convergence. The algorithm we propose is similar, in structure, to iteration presented in (9), so the same definitions hold except for the step size , which is considered to be fixed here. In fact, the paracontraction property of the employed operator in proposed algorithm, allows us to prove the convergence, even with the fixed . Further, we will show in Section 6 via numerical simulations that the algorithm actually performs faster with an appropriate choice of fixed step size .
Let the elements of the iteration, i.e., the set of agents , the operator , the vector and the matrix be as in (9), defined in Subsection 3.2. Then, the distributed allocation procedure on time varying networks, takes the form:

(11)

Note that, in our proposed iteration in (11), there are two differences compared to (9). First, the step size is fixed and secondly the elements of communication matrix can take values from finite set. The latter implies that there are finite number of adjacency matrices available, for the communication among agents. Formally, Each element of communication matrix , i.e., , can take the values in a finite set. We can also redefine the operator in (10) with fixed as to write (11) in compact form as:

(12)

Note that, because of fixed step size in (11) and Assumption 5, the operator sequence will belong to a finite family of paracontractions. This will allow us to exploit the following well-known theorem, proved by Elsner et al. (1992), later for our convergence result. (Elsner et al. (1992)) Let be a finite family of paracontractions such that , and consider the iteration

where, for each . Then, the state converges to a common fixed point of the paracontractions that occur infinitely often in the sequence.

We now have the necessary tools and algorithmic setup to show, in the following theorem, that iteration in (11)(12) converges to a consensus vector, see in (2) which belongs to CORE, in (1). Let and the operator be a paracontraction with in (1). Let Assumptions and hold. Then, the iteration in (12) is such that:

  1. is a sequence of time-varying paracontractions;

  2. ;

  3. for some ,

where is the CORE set (1) and is the consensus set (2). (i): It follows directly from Remark 4.2 on Corollary 4.2.
(ii): To characterise the fixed points of in (12), let i.e. . And let . Here, we want to show that .
It follows by Perron-Frobenius theorem that , regardless of the temporal variation in . So, . Consequently, . And, as , hence , which concludes the proof of this assertion.
(iii): It follows from assertion (i), (ii), Assumption 5 and direct application of Lemma 5. This result shows a remarkable ability of operator-theoretic tools to describe algorithms in general form. For instance, our algorithm in (10) allows a mechanism designer to choose an operator in (11) of his choice to possibly steer the consensus towards a particular point in set in (1). This operator is primarily required to fulfill two necessary requirements: it should be a paracontraction and .

(a)
(b)
Figure 3: (a): The trajectories of with (b): The trajectories of with .

6 Numerical Simulations

In our numerical simulations, we consider a coalitional game played among agents with a set of agents as . Coalitions, including the singleton, are assigned with a value specified by characteristic function . We set, . Now, a payoff vector, as in Definition 2, that belongs to CORE, in (1) must allocate each agent at-least its individual value, sum of their allocations should be and be group rational. Consistent with these requirements, the CORE of this game is the following set:

The agents communicate over time-varying graphs associated with the adjacency matrices . Here, we set the adjacency matrices to be:


for all . Note that this graph sequence satisfies Assumption 3.2 with , and the elements of the adjacency matrices satisfy Assumption 3.2 with
For the initial assignments, we assume that each agent allocates entire value of coalition, i.e., to itself. For example, the initial proposal by agent 1 will be . Finally, we apply the iteration in (11) with the operator , and as expected, the local allocations converge to .
In Figure 3(a), we compare the trajectories of normalized distances , by varying for a specified . We can observe that a higher value of corresponds to a faster convergence. In Figure 3(b), we use the same metric and observe the convergence speed while varying . As expected, the convergence of iteration with fixed step size , is faster compared to a decreasing sequence as in [Bauso and Notarstefano (2015)].

7 Conclusion

We presented a partial operator-theoretic characterization of the approachability principle and showed that it contains a paracontraction operator. Based on this result, we have proposed a distributed payoff allocation algorithm, with fixed step sizes, and proved its convergence via operator-theoretic arguments. Such analysis of algorithms, based on operator theory, allow more general description of their structure and hence open further improvement possibilities.
As future work, we aim to completely characterize the approachability principle in operator-theoretic terms. It would also be valuable to relax the assumption on the communication graph from double stochasticity to row stochasticity.

References

  • D. Bauso, M. Cannon, and J. Fleming (2014) Robust consensus in social networks and coalitional games. IFAC Proceedings Volumes 47 (3), pp. 1537–1542. Cited by: §1.
  • D. Bauso and G. Notarstefano (2015) Distributed n-player approachability and consensus in coalitional games. IEEE Transactions on Automatic Control 60 (11), pp. 3107–3112. External Links: Document, 1310.1874, ISSN 00189286 Cited by: §1, §1, §3.1, §3.2, §3.2, §4.2, §4.2, §6.
  • D. Blackwell (1954) An analog of the minimax theorem for vector payoffs. Pacific Journal of Mathematics 6 (1), pp. 1–8. External Links: Document, ISSN 00308730 Cited by: §1, §3.1.
  • L. Elsner, I. Koltracht, and M. Neumann (1992) Convergence of sequential and asynchronous nonlinear paracontractions. Numerische Mathematik 62 (1), pp. 305–319. External Links: Document, ISSN 0029599X Cited by: §4.1, §5.
  • Ernest Ryu and S. Boyd (2016) a Primer on Monotone Operator Methods. Applied and computational mathematics 15 (1), pp. 3–43. Cited by: §1, §4.1.
  • F. Fele, J. M. Maestre, and E. F. Camacho (2017) Coalitional control: cooperative game theory and control. IEEE Control Systems Magazine 37 (1), pp. 53–69. Cited by: §1.
  • D. Fullmer and A. S. Morse (2018) A Distributed Algorithm for Computing a Common Fixed Point of a Finite Family of Paracontractions. IEEE Transactions on Automatic Control 63 (9), pp. 2833–2843. External Links: Document, ISSN 15582523 Cited by: §4.2, §4.2.
  • L. Han, T. Morstyn, and M. McCulloch (2018) Incentivizing prosumer coalitions with energy management using cooperative game theory. IEEE Transactions on Power Systems 34 (1), pp. 303–313. Cited by: §1.
  • E. Lehrer (2003) Allocation processes in cooperative games. International Journal of Game Theory 31 (3), pp. 341–351. Cited by: §1, §3.1.
  • M. Maschler, B. Peleg, and L. S. Shapley (1971) The kernel and bargaining set for convex games. International Journal of Game Theory 1 (1), pp. 73–93. Cited by: §1.
  • A. Nedich and D. Bauso (2013) Dynamic coalitional TU games: Distributed bargaining among players’ neighbors. IEEE Transactions on Automatic Control 58 (6), pp. 1363–1376. External Links: Document, ISSN 00189286 Cited by: §1.
  • W. Saad, Z. Han, M. Debbah, A. Hjørungnes, and T. Başar (2009) Coalitional game theory for communication networks. IEEE Signal Processing Magazine 26 (5), pp. 77–97. External Links: Document, ISSN 10535888 Cited by: §1.
  • W. Saad, Z. Han, M. Debbah, and A. Hjørungnes (2008) A Distributed Merge and Split Algorithm for Fair Cooperation in Wireless Networks. IEEE Transactions on Wireless Communications 8 (9), pp. 4580–4593. External Links: Document, 0802.2159, ISBN 9781424420513, ISSN 1536-1276 Cited by: §1.
  • W. Saad, Z. Han, and H. V. Poor (2011) Coalitional Game Theory for Cooperative Micro-Grid Distribution Networks. In 2011 IEEE International Conference on Communications Workshops (ICC), pp. 1–5. External Links: Document, ISBN 978-1-61284-954-6 Cited by: §1.
  • H. E. Scarf (1967) The core of an n person game. Cited by: §1.
  • D. Schmeidler (1969) The nucleolus of a characteristic function game. SIAM Journal on applied mathematics 17 (6), pp. 1163–1170. Cited by: §1.
  • L. S. Shapley (1953) A value for n-person games. Contributions to the Theory of Games 2 (28), pp. 307–317. Cited by: §1.
  • M. Smyrnakis, D. Bauso, and H. Tembine (2019) Game-theoretic learning and allocations in robust dynamic coalitional games. SIAM Journal on Control and Optimization 57 (4), pp. 2902–2923. Cited by: §1.
  • J. M. Zolezzi and H. Rudnick (2002) Transmission cost allocation by cooperative games and coalition formation. IEEE Transactions on power systems 17 (4), pp. 1008–1015. Cited by: §1.