Distributed and Efficient Resource Balancing Among Many Suppliers and Consumers

I Introduction

Many modern problems involve one resource that is generated over time by a number of suppliers and simultaneously consumed by a number of consumers. For instance, consider electrical power generated or injected into the grid by utility companies, wind mills, solar panels, batteries, and electric vehicles. At the same time, other devices consume the same power. Other examples include cryptocurrency markets with large numbers of producers and consumers, etc.

These problems are complex problems that involve a huge number of decision-makers. Each producer has a different cost for producing a unit of the resource, and each consumer derives a different benefit from consuming the resource. One desirable outcome of the interaction between producers and consumers is to maximize the social welfare or minimize the social cost. If we can represent each agent by an utility function, then we aim to maximize the sum of utilities of producers and consumers subject to capacity constraints on the quantities. These constraints ensure that quantities are nonnegative and that the total consumption is at most the total production.

It is well-known that in the presence of money and price signals, an equilibrium can be reached through a tatonnement process where social welfare is maximized. However, we propose a mechanism where neither money, nor price signals, are required to maximize social welfare: we only require binary feedback signals on whether or not total consumption exceeds total production. Our mechanism is decentralized: each agent only requires knowledge of its own utility function. We show that if every agent follows a specified algorithm, then over time, the profile of decisions (production and consumption quantities) converges to a socially optimal outcome.

This problem can be related to the TCP congestion control problem, where data sources controlling their own rates can interact to achieve an optimal network-wide rate allocation, where supplier agents can act as source of allocation, while consumer agents act as the consumers of the allocated resources. The detailed analysis [1] of increase/decrease algorithms used predominantly in the past for TCP congestion control, which include Multiplicative Increase/Multiplicative Decrease (MIMD) [2], Additive Increase/Additive Decrease (AIAD) [3], Additive Increase/ Multiplicative Decrease (AIMD) [4], and Multiplicative Increase/Additive Decrease (MIAD), shows that AIMD leads to the most optimal resource allocation.

AIMD as a feedback control algorithm has been investigated a lot in the literature [5, 6, 7, 8, 9]. In this setting, each agent determines its own allocation as per the AIMD algorithm. Similar distributed optimization algorithms have been shown to iteratively converge to an optimal allocation of resources [10, 11, 12], but these algorithms rely on inter-agent communication to achieve optimality. While a combined version of the classical AIMD algorithm with a probabilistic rule [13]

defines the behavior of multiple agents in response to a capacity signal. This non-homogeneous Markov chain is showed to reach sure convergence, to the social optimum. Here, each agent responds to a capacity event according to its own probability function, known only to that agent, without the need of any communication with another agent. In this sense,

[13] goes beyond traditional AIMD and emulates RED-like congestion control [14]. Also, very limited actuation is assumed for an agent, and the agent only decides to respond to a capacity event or not, in an asynchronous manner. There is no need for a common clock and the setting is completely stochastic.

More generally, problems concerning multiple independent producers and consumers can occur in many different domains of application—for instance, electrical power [15], solar energy and microgrids [16], data centers [17], and air cargo [18]

. Such problems can be considered as specific instances of machine learning

[19], or as problems of collective control over distributed dynamical systems [20].

A simple strategy of rational learning where each producer and each consumer acts based on the previous instance, is unsatisfactory [21]. Many classes of equilibrium problems are computationally hard [22].

This paper is organized as follows. First, we describe the mathematical model in Section II. We describe our proposed solution approach in Section III. We show the optimality of our approach in Section IV and describe the iterative convergence to optimality through simulations in Section V. Finally, we sum up our contributions and conclude in Section VI.

Ii Setting

We consider a market with a single good (arbitrarily divisible), with multiple suppliers and multiple consumers. Three types of convergence are routinely observed. First, the total demand and total supply converge through a dynamic process that requires the exchange of information on the price of a good, the total demand, or the total supply (e.g., through a process of tatonnement). In the presence of multiple suppliers, these suppliers may compete via their production quantities, which is known as Cournot competition. In Cournot competition, the production quantity of each supplier may converge as well through a tatonnement process. Similarly, in the presence of multiple consumers, these consumers may also compete for the limited quantity of good available through bidding and proportional allocation (e.g., competition among newsvendors). Here again, we observe convergence of the allocated quantities of each consumer. In these situations, the convergence relies on an exchange of information in the form of prices, bids, and quantities of goods. In this work, we consider a situation where communication, e.g., the exchange of information, is limited.

The notation that we use throughout the paper is as follows. There is a group of suppliers $S$ , and a group of consumers $C$ . Time is discrete: $t = 1, 2, \dots$ For each $i \in S$ , the amount of good supplied at time $t$ is $x_{i} (t)$ . For each $j \in C$ , the amount of good consumed at time $t$ is $y_{j} (t)$ . The total supply of good at time $t$ is $x (t)$ , the total consumption is $y (t)$ . For each $i \in S$ , the utility function $f_{i} (z)$ represents the profit from producing an amount $z$ . For each $j \in C$ , the utility function $g_{j} (z)$ represents the profit from consuming an amount $z$ . Our objective is twofold. On one hand, we want to see the individual consumption quantities and production quantities to converge using limited inter-agent communication. On the other hand, we want the limit to be efficient: the total supply and total demand are balanced, the total supply is allocated optimally among suppliers, and the total demand is allocated optimally among the consumers.

Remark 1 (Choice of utility function).

Whether the agent wants to respond to the capacity signal or not, highly depends on the choice of the utility function for that agent. The utility function has to be selected, considering the fact that it should lead to a maximum so that we can reach an optimal point, which maximizes profit for the agent. Therefore, we consider concave functions, which will be demonstrated to reach a global maximum at the optimum quantity supplied/consumed, in the later sections.

More precisely, the first objective requires that there exist constants ${x_{i}^{*}}$ and ${y_{i}^{*}}$ such that for every $i$ and $j$ , we have

	$x_{i} (t) \to x_{i}^{*},$		(1)
	$y_{j} (t) \to y_{j}^{*},$		(2)

as $t \to \infty$ . The second objective requires that ${x_{i}^{*}}$ and ${y_{i}^{*}}$ satisfy the following:

	$(x^{}, y^{}) \in max x, y$	$\sum i \in S f_{i} (x_{i}) + \sum j \in C g_{j} (y_{j})$		(3)
	such that	$\sum i \in S x_{i} = \sum j \in C y_{j} .$		(4)

The limited communication property will be presented after presenting the algorithms for computing ${x_{i} (t), y_{j} (t)}$ .

For the sake of discussion, and as a baseline, we also define the following vectors

u^{*}

and

w^{*}

solving the following unconstrained optimization problems:

u^{*} \in arg max x \in R^{S}

\sum i f_{i} (x_{i}),

(5)

and

w^{*} \in arg max y \in R^{C}

\sum j g_{j} (y_{j}) .

(6)

Next, we present the proposed distributed algorithm that specifies how agents update their $x_{i} (t)$ and $y_{j} (t)$ over time.

Iii Algorithm

We consider a distributed environment, where we have a total of S suppliers and C consumers, such that each time instant $t$ , each supplier $i$ supplies an amount $x_{i} (t)$ and after that, each consumer $j$ consumes an amount $y_{j} (t)$ .

The procedures for updating $x_{i} (t)$ and $y_{j} (t)$ are presented in Algorithm 1 and Algorithm 2.

At the time instant $t$ , we keep a check whether the total amount supplied at $(t - 1)$ was equivalent to the total amount consumed, or not. If the supply was more than the consumption, a capacity signal is sent to all the supplier agents to reduce the supply amount for $t$ . Otherwise, if the consumption was more than the supply, a capacity signal is sent to the consumer agents to reduce the consumption for $t$ . This makes sure that an equilibrium is maintained throughout the execution, whenever the supply or consumption fluctuates.

If the agent responds to the capacity signal, it reduces the amount in the next time instant by a factor of $β$ , or else, it keeps on increasing the amount after every time instant by adding the value of $α$ to it. When the agent receives a capacity signal, its probability to reduce by a factor of $β$

depends on the Bernoulli random variable

b_{i} (t)

, as

P (b_{i} (t) = 1) = λ

, and

P (b_{i} (t) = 0) = 1 - λ

An agent also keeps track of its individual long term average, which is given by:

_{i} (t) = (\frac{1}{t + 1}) t \sum T = 0 x_{i} (T) .

We want to consider limited inter-agent communication. The only communication available at time $t$ are the signals:

	$s (t)$	$≜ 1_{[x (t - 1) < y (t - 1)]}$
		$= 1_{[\sum_{i} x_{i} (t - 1) < \sum_{j} y_{j} (t - 1)]} t = 1, 2, \dots$		(7)

1: Input:

t

f_{i}

x_{i} (t - 1)

s (t)

Γ

α_{s}

and

β_{s}

2: Output:

x_{i} (t)

3: Initialization:

λ = 0

and

_{i} (t) = 0

4: if (

s (t) = 1

) then

λ = Γ (f_{i}^{'} (_{i} (t - 1)) /_{i} (t - 1))

6: generate independent Bernoulli random variable

b_{i} (t)

with parameter

λ

7: end if

8: if (

b_{i} (t) = 1

) then

x_{i} (t) = x_{i} (t - 1) β_{s}

10: else if

x_{i} (t - 1) \leq {x_{i}}^{*}

then

11:

x_{i} (t) = x_{i} (t - 1) + α_{s}

12: else if

x_{i} (t - 1) > {x_{i}}^{*}

then

13:

x_{i} (t) = x_{i} (t - 1) - α_{s}

14: end if

115:

_{i} (t) = ((_{i} (t - 1) \cdot (t - 1)) + x_{i} (t)) / (t + 1)

;

16: return

x_{i} (t)

Algorithm 1 AIMD Algorithm Supplier

i

1: Input:

t

g_{j}

y_{j} (t - 1)

c (t)

Γ

α_{c}

and

β_{c}

2: Output:

y_{j} (t)

3: Initialization:

λ = 0

and

_{j} (t) = 0

4: if (

c (t) = 1

) then

λ = Γ (g_{j}^{'} (_{j} (t - 1)) /_{j} (t - 1))

6: generate independent Bernoulli random variable

b_{j} (t)

with parameter

λ

7: end if

8: if (

b_{j} (t) = 1

) then

y_{j} (t) = y_{j} (t - 1) β_{c}

10: else if

y_{j} (t - 1) \leq {y_{j}}^{*}

then

11:

y_{j} (t) = y_{j} (t - 1) + α_{c}

12: else if

y_{j} (t - 1) > {y_{j}}^{*}

then

13:

y_{j} (t) = y_{j} (t - 1) - α_{c}

14: end if

15:

_{j} (t) = ((_{j} (t - 1) \cdot (t - 1)) + y_{j} (t)) / (t + 1)

16: return

y_{j} (t)

Algorithm 2 AIMD Algorithm Consumer

j

The pseudocode for the procedure describing what happens at the supplier side is shown in Algorithm 1. Each agent maintains the last value sent, $x_{i} (t - 1)$ and the long term average of the values sent in the past, $_{i} (t)$ , and this along with the concave nature of its utility function helps the agent drive the quantity sent towards the optimal point, ${x_{i}}^{*}$ .

At the iteration $t$ of the algorithm, it is first checked whether the total quantity sent at $(t - 1)$ was more than the total quantity consumed(condition for capacity signal being sent to the agent), or not (line 4) for each of the supplier agent. If the condition is satisfied, probability $λ$ is calculated (line 5) as a function of utility derivative and long-term average, for $(t - 1)$ . On the basis of this probability, it is determined whether the agent responds to the capacity signal or not. The agent responds to the signal by reducing the quantity sent at $t$ by a factor of $β$ , in comparison to what it sent at $(t - 1)$ (line 9). This depends on the independent Bernoulli random variable $b_{i} (t)$ with parameter $λ$ (line 6).

Else, it checks whether the value supplied at $(t - 1)$ was lesser than the optimum point of the agent’s utility function or not (line 10). If satisfied, the agent adds the value of $α$ to quantity supplied at $(t - 1)$ and send it at $t$ (line 11). Otherwise, it sends the quantity by subtracting $α$ to the quantity sent at $(t - 1)$ (line 13). This condition is included because we do not want the agent to over-supply, and also makes sure that if this agent has reached its optimum supply quantity, other agents follow suit and are driven towards their individual optimum supply quantities. We will prove this statement later.

Then, the long term average is calculated (line 15), and the quantity is supplied by agent $i$ at $t$ (line 16). The same algorithm runs at the consumer side (Algorithm 2), and both of these concurrent running algorithms make sure that our system reaches the state which leads to maximum combined profit.

Iv Convergence

In this section, we show the convergence over time of the sequences of produced quantities ${x_{i} (t)}$ for all producers, as well as the sequences of consumed quantities ${y_{i} (t)}$ for all consumers.

Theorem 1 (Convergence Theorem).

Suppose that every function $f_{i}$ and $g_{j}$ is concave and achieves its maximum at a finite point. For every producer $i$ and every consumer $j$ , we have

x_{i} (t) \to u_{i}^{*}, y_{j} (t) \to w_{j}^{*} .

Proof Sketch.

Recall $u^{*}$ and $w^{*}$ are defined in (5,6). The proof proceeds by three steps. First, we show that $\sum_{j} y_{j} (t)$ converges as $t \to \infty$ . Secondly, we show that $x_{i} (t)$ converges for all $i$ . Lastly, we show that $y_{j} (t)$ converges for all $j$ .

Step 0. Observe that since each $g_{j}$ achieves its maximum at the point $w_{j}^{*}$ , hence, there exists a finite number $C_{y}$ such that $\sum_{j} w_{j}^{*} = C_{y}$ . Observe that, if $\sum_{j} y_{j} (t)$ converges, the limit must be $C_{y}$ .

Step 1. Consider the sequence

^s (t) = 1_{[\sum_{i} x_{i} (t - 1) < C_{y}]} .

By continuity, the AIMD algorithm with input $^s (t)$ and the AIMD algorithm with input $s (t)$ converge to the same limit point. Therefore, by [13, Theorem 1], we have $x_{i} (t) \to u_{i}^{*}$ for all $i$ .

Step 2. Consider the sequence

^c (t) = 1_{[\sum_{i} u_{i}^{*} < \sum_{j} y_{j} (t - 1)]} .

Since $x_{i} (t) \to u_{i}^{*}$ for all $i$ , by continuity, the AIMD algorithm with input $^c (t)$ and the AIMD algorithm with input $c (t)$ converge to the same limit point. Therefore, by [13, Theorem 1], we have $y_{j} (t) \to w_{j}^{*}$ for all $i$ . ∎

V Simulations

In this section, we simulate the interactions of suppliers and consumers in two settings; first, when their utility functions are non-monotonic concave, as in Figure 0(a), second, when the supplier utility functions are monotonic concave, as in Figure 4(a).

V-a Non-monotonic Utility Functions

We simulate a total of 9 supplier agents, and 18 consumer agents. We generate random utility functions ${f_{i}, g_{j}}$ for the agents, while ensuring that the following sums on the supplier and consumer sides are deterministic and equal:

\sum i \in S max z_{i} f_{i} (z_{i}) = \sum j \in C max z_{j} g_{j} (z_{j}) = 900.

(8)

The values of $α$ and $β$ for supplier and consumer agents is 5 and 0.75 respectively. The network constant $Γ$ is kept at 2.0 to ensure that the probability $λ$ remains in the interval $[0, 1]$ .

(a) Sum of Supplier and Consumer utilities with respect to the change in total supply/consumption

(a) Average supplied amount by the 5 agents driven towards their optimum point

(a) Average consumed amount by randomly selected 5 consumer agents, driven towards their optimum point

The total supply and consumption lingers around the peak total utility, as shown in (8), i.e. $900$ . As seen in Figure 1(b), the total supply from all supplier agents is balanced by the total consumption by all consumer agents.

From Figures 2(a) and 3(a), we can see that the respective supplier and consumer agent long-term averages saturate around their respective optimum points. Figures 2(b) and 3(b) show the utility-function derivatives converging towards 0 for both supplier and consumer agents, depicting the maximum profit for each agent, while Figure 1(c) shows the 95% confidence on the supplier utility derivative. The same can be understood by examining Figure 1(a), that the sum of all the utilities converges towards optima, i.e. 900, which was assumed as a constant in the beginning of our simulation.

V-B Monotonic Supplier Utility Functions

Now we consider a scenario where suppliers have a monotonic non-decreasing utility function, while the consumers have concave utility. So, the supplier utility (see Figure 4(a)) is of the form

f_{i} (_{i} (t)) = ℓ_{i} \sqrt{_{i} (t)}

(9)

while consumer utilities (Figure 0(b)) are of the form

g_{j} (_{j} (t)) = - \frac{(_{j} (t) - {y_{j}}^{*})^{2}}{ℏ_{j}} + (1 .5) ℏ_{j}

(10)

On simulating for the same number of supplier and consumer agents, along with the same constants as in the previous simulation, we realize that sum of quantities supplied and consumed tend to saturate around the optimal sum (see Figure 4(c)), even when no particular maximum is present for the supplier utility. The utility sum for consumers does converge towards the optimal sum (see Figure 4(b)), unlike the supplier utility’s sum, as there is no particular maximum for the monotonic non-decreasing functions.

Vi Conclusion

In this paper, we have utilized the convergence properties of the AIMD algorithm [13], to solve the problem of maintaining an equilibrium of supply and demand, for a group of distributed agents [21], by also maximizing their respective profits.

As showed in our simulations (see Section V-A), for a concave type utility functions for both supplier and consumer agents, i.e., with utilities which have clear maxima, profit-maximization and equal sum of supply and demand holds true. By other simulations (see Section V-B), even when either side—supply or demand—does not have a concave utility, equilibrium of supply and demand is satisfied, along with the profit-maximization of the agent with concave utility.

Considering the slew of applications requiring a global, dynamic balance between supply and demand, such as the management of data centers, energy producers and consumers connected to a smart grid, and the like, we surmise that the work presented here can be put to profitable use in several domains of application.

Acknowledgment

Jia Yuan Yu was supported by the Natural Sciences and Engineering Research Council of Canada (RGPIN-2018-05096).

References

[1] D.-M. Chiu and R. Jain, “Analysis of the increase and decrease algorithms for congestion avoidance in computer networks,” Computer Networks and ISDN systems, vol. 17, no. 1, pp. 1–14, 1989.
[2] E. Altman, K. Avrachenkov, C. Barakat, A. A. Kherani, and B. Prabhu, “Analysis of mimd congestion control algorithm for high speed networks,” Computer Networks, vol. 48, no. 6, pp. 972–989, 2005.
[3]
K. Xu and N. Ansari, “Stability and fairness of rate estimation-based AIAD congestion control in TCP,”
IEEE Commun. Lett., vol. 9, no. 4, pp. 378–380, 2005.
[4] Y. R. Yang and S. S. Lam, “General AIMD congestion control,” in 8th International Conference on Network Protocols (ICNP 2000), pp. 187–198, IEEE, 2000.
[5] S. Jacobs and A. Eleftheriadis, “Providing video services over networks without quality of service guarantees,” in World Wide Web Consortium Workshop on Real-Time Multimedia and the Web, 1996.
[6] R. Rejaie, M. Handley, and D. Estrin, “Rap: An end-to-end rate-based congestion control mechanism for realtime streams in the internet,” in INFOCOM’99. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies, vol. 3, pp. 1337–1345, IEEE, 1999.
[7] D. Sisalem and H. Schulzrinne, “The loss-delay based adjustment algorithm: A TCP-friendly adaptation scheme,” in Proceedings of NOSSDAV, vol. 98, pp. 215–226, Citeseer, 1998.
[8] S. Cen, J. Walpole, and C. Pu, “Flow and congestion control for internet media streaming applications,” in Multimedia Computing and Networking 1998, vol. 3310, pp. 250–265, International Society for Optics and Photonics, 1997.
[9] M. Corless, C. King, R. Shorten, and F. Wirth, AIMD dynamics and distributed resource allocation, vol. 29. SIAM, 2016.
[10] S. S. Ram, A. Nedić, and V. V. Veeravalli, “Distributed stochastic subgradient projection algorithms for convex optimization,” Journal of optimization theory and applications, vol. 147, no. 3, pp. 516–545, 2010.
[11] A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. Autom. Control, vol. 54, no. 1, pp. 48–61, 2009.
[12] J. C. Duchi, A. Agarwal, and M. J. Wainwright, “Dual averaging for distributed optimization: Convergence analysis and network scaling,” IEEE Trans. Autom. Control, vol. 57, no. 3, pp. 592–606, 2012.
[13] F. Wirth, S. Stuedli, J. Y. Yu, M. Corless, and R. Shorten, “Nonhomogeneous Place-Dependent Markov Chains, Unsynchronised AIMD, and Network Utility Maximization,” arXiv 1404.5064 [math.OC], Apr. 2014.
[14] R. Srikant, “Internet congestion control, vol. 14 of control theory,” 2004.
[15] A. Muralidharan, H. A. Maior, and S. Rao, A Self-Governing and Decentralized Network of Smart Objects to Share Electrical Power Autonomously, pp. 25–48. Cham: Springer International Publishing, 2018.
[16] I. Maity and S. Rao, “Simulation and pricing mechanism analysis of a solar-powered electrical microgrid,” IEEE Syst. J., vol. 4, pp. 275–284, Sept. 2010.
[17] N. Singh and S. Rao, “Ensemble learning for large-scale workload prediction,” IEEE Trans. Emerg. Topics Comput., vol. 2, pp. 149–165, June 2014.
[18] R. Totamane, A. Dasgupta, and S. Rao, “Air cargo demand modeling and prediction,” IEEE Syst. J., vol. 8, pp. 52–62, Mar. 2014. doi:10.1109/JSYST.2012.2218511.
[19] T. M. Mitchell, Machine Learning. McGraw Hill, 1997.
[20] D. H. Wolpert, K. R. Wheeler, and K. Tumer, “Collective intelligence for control of distributed dynamical systems,” Europhysics Letters, vol. 49, no. 6, p. 708, 2000.
[21] P. K. Enumula and S. Rao, “The Potluck Problem,” Economics Letters, vol. 107, pp. 10–12, Apr. 2010.
[22] B. Codenotti, B. Mccune, S. Pemmaraju, R. Raman, and K. Varadarajan, “An experimental study of different approaches to solve the market equilibrium problem,” ACM Journal on Experimental Algorithmics, vol. 12, pp. 3.3:1–3.3:21, Aug. 2008.

Distributed and Efficient Resource Balancing Among Many Suppliers and Consumers

Authors

Distributed Multi-resource Allocation with Little Communication Overhead

Multi-Layers Supply chain modelling based on Multi-Agent Approach

A Deep Q-Network for the Beer Game with Partial Information

Optimizing Supply Chain Management using Gravitational Search Algorithm and Multi Agent System

Multi-resource allocation for federated settings: A non-homogeneous Markov chain model

Design of Intelligent Agents Based System for Commodity Market Simulation with JADE

Matching Demand with Supply in the Smart Grid using Agent-Based Multiunit Auction

I Introduction

Ii Setting

Remark 1 (Choice of utility function).

Iii Algorithm

Iv Convergence

Theorem 1 (Convergence Theorem).

Proof Sketch.

V Simulations

V-a Non-monotonic Utility Functions

V-B Monotonic Supplier Utility Functions

Vi Conclusion

Acknowledgment

References

Distributed and Efficient Resource Balancing Among Many Suppliers and Consumers

Authors

Related Research

Distributed Multi-resource Allocation with Little Communication Overhead

Multi-Layers Supply chain modelling based on Multi-Agent Approach

A Deep Q-Network for the Beer Game with Partial Information

Optimizing Supply Chain Management using Gravitational Search Algorithm and Multi Agent System

Multi-resource allocation for federated settings: A non-homogeneous Markov chain model

Design of Intelligent Agents Based System for Commodity Market Simulation with JADE

Matching Demand with Supply in the Smart Grid using Agent-Based Multiunit Auction

This week in AI

I Introduction

Ii Setting

Remark 1 (Choice of utility function).

Iii Algorithm

Iv Convergence

Theorem 1 (Convergence Theorem).

Proof Sketch.

V Simulations

V-a Non-monotonic Utility Functions

V-B Monotonic Supplier Utility Functions

Vi Conclusion

Acknowledgment

References