I Introduction
With the widescale proliferation of cloud computing, a large number of individuals and businesses have transitioned towards the use of cloud computing platforms [1]. Users (applications)^{1}^{1}1Throughout the paper, we use the term “users” and “applications” interchangeably. request resources from a cloud service provider for a particular task and service provider needs to provide requested resources based on a Service Level Agreement (SLA). The goal for any service provider is to satisfy as many applications as possible by a timely provision of resources, resulting in an increase in user satisfaction as well as utility of service providers. However, the provider often lacks sufficient available resources and thus allocation of resources to different users is not straightforward.
Optimal allocation of resources for cloud computing is a fundamental research problem [2, 3, 4]. Due to the recent advent of paradigms such as Big Data [5], Internet of Things (IoT) [6], and Machine to Machine (M2M) communication, the problem is further exacerbated as the demand for limited resources has increased further. Even when resources are available, allocating them optimally to different applications requires efficient algorithms. Hence, it is important to ascertain the availability of resources and allocate them optimally to different applications.
To ensure the availability of resources, numerous solutions have been proposed in the literature ranging from buying more resources to meet peak demands, to creating shared resource pools among geographically colocated service providers. However, different service providers may have different objectives and existing resource pool based mechanisms do not consider utilities of different service providers. Ideally, a resource sharing mechanism should consider objectives of different service providers, which results in a multiobjective optimization (MOO) problem. Furthermore, while resource sharing or creation of a resource pool may tackle the resource availability problem, it does not address resource allocation problem, i.e., optimally allocating the available resources to different users. Therefore, there is a need for a framework that not only ascertains resource availability by enabling resource sharing among different service providers, but also optimally allocates resources to users, while accounting for the service providers’ objectives.
We present a cooperative game theory based framework that can be used for resource sharing and allocation in cloud computing. We allow resource sharing among different cloud service providers, while considering the objectives (utilities) of cloud service providers. Rather than considering a specific resource, we consider a wide range of resources that are present across different cloud service providers. We show that different resource providers can benefit by creating a coalition among service providers and then allocating resources to users. We also show that the coalition of service providers is stable, where no service provider will incur any loss (when compared with working alone). Also, we obtain a Pareto optimal (details in Section II) solution to our multiobjective resource sharing problem.
The main contributions of this paper are:

We propose a cooperative game theory (CGT) based multiobjective resource sharing and allocation framework for cloud computing that guarantees Pareto optimality and consider different objectives of the service providers that share their resources.

We show that the resource sharing and allocation problem can be modeled as a canonical game with nontransferable utility. Furthermore, when service provider objectives are represented by monotone nondecreasing utilities, we show that our canonical game is convex. Due to the convexity of game, the core is nonempty and the grand coalition formed by all service providers is stable.

We address the problem of obtaining an allocation from the core by proposing a computationally efficient algorithm that is guaranteed to provide an allocation from the core.

We evaluate the performance of our proposed framework through extensive experiments and show that resource sharing can improve utility of all service providers and also increase user satisfaction.
The paper is further structured as: Section II provides a primer on multiobjective optimization, cooperative game theory, canonical games, and the core. Section III presents the system model used in this paper. It also presents the resource sharing and allocation problem along with theoretical properties of the cooperative game used to obtain the Pareto optimal solution. Section IV presents our experimental results. Section V provides a review of different solutions proposed in literature for resource allocation, while Section VI concludes the paper.
Ii Preliminaries
In this section, we present a discussion on multiobjective optimization, cooperative game theory, and the core.
Iia MultiObjective Optimization
For inequality constraints and
equality constraints, MOO identifies a vector
that optimizes a vector function(1) 
such that
(2)  
where is a vector of decision variables and the feasible set is denoted by . The fundamental difference between a single objective optimization (SOO) and MOO is that MOO involves a vector of objective functions rather than a single objective function. Therefore, in MOO, the optimal solution is not a single point but a frontier of solutions known as Pareto frontier or Pareto boundary (see [7] for details). Some basic definitions related to MOO are:
Definition 1.
Pareto Optimality: For any maximization problem, is if the following holds for every ,
(3) 
where and .
Figure 1 shows a MOO problem (minimization problem) with two objective functions and . The boundary is the Pareto frontier as it consists of all the Pareto optimal solutions.
IiB Cooperative Game Theory
Cooperative game theory provides a set of analytical tools that assists in understanding the behavior of rational players in a cooperative setting [8]. Players can have agreements among themselves that affect the strategies as well as obtained utilities of game players. Coalition games are one of the basic types of cooperative games that deal with the formation of coalitions, namely groups of two or more cooperating players. Formally,
Definition 2.

is a closed and convex subset of .

is comprehensive, i.e., if we are given payoffs and where , then . In other words, if the members of coalition can achieve a payoff allocation , then the players can change their strategies to achieve an allocation .

The set and , with is a bounded subset of . In other words, the set of vectors in for a coalition where the coalition members receive a payoff at least as good as working alone (noncooperatively) is a bounded set.
Definition 3.
Value of a coalition: The sum of all players’ payoff from a particular payoff vector for any coalition is known as the value of a coalition.
It is worth mentioning that is the set of payoff vectors, while is the sum of payoffs that all players get in a particular payoff vector (coalition).
Definition 4.
NonTransferable Utility (NTU)[8]: If the total utility of any coalition cannot be assigned a single real number or if there is a rigid restriction on utility distribution among players, then the game has a nontransferable utility.
Definition 5.
Characteristic function[8]: The characteristic function for any coalition game with NTU is a function that assigns a set of payoff vectors, , where each element of the payoff vector represents a payoff that player obtains, depending on the selected strategy, within the coalition .
Definition 6.
Characteristic form Coalition Games [8]: A coalition game is said to be of characteristic form, if the value of coalition depends only on the members of coalition.
Definition 7.
Superadditivity of NTU games[9]: A canonical game with NTU is said to be superadditive if the following property is satisfied.
(4) 
i.e., if any two disjoint coalitions and form a large coalition , then the coalition can always give its members the payoff that they would have received in the disjoint coalition and .
Definition 8.
Canonical Game: A coalition game is canonical if it is superadditive and in characteristic form.
The core is a widely used solution concept for canonical games as discussed below.
IiC Core
Definition 9.
Group Rational: A payoff vector is grouprational if .
Definition 10.
Individually Rational: A payoff vector is individuallyrational if every player can obtain a payoff no less than acting alone, i.e., .
Definition 11.
Imputation: A payoff vector that is both individually and group rational is an imputation.
Definition 12.
Grand Coalition: The coalition formed by all game players in is the grand coalition.
Based on the above definitions, we can now define the core of an NTU canonical coalition game.
Definition 13.
Core[8]: For any NTU canonical game , the core is the set of imputations in which no coalition has any incentive to reject the proposed payoff allocation and deviate from the grand coalition to form a coalition instead. This can be mathematically expressed as
(5) 
Remark 1.
Any payoff allocation from the core is Paretooptimal as evident from definition of the core. Furthermore, the grand coalition formed is stable, i.e., no two players will have an incentive to leave the grand coalition to form a smaller coalition.
However, the core is not always guaranteed to exist. Even if the core exists, it may be very large as it a convex set. Therefore, finding a suitable allocation from the core is challenging.
Iii System Model
Let be the set of all the cloud service providers that act as players in our game. We assume that each player has a set of different types of resources such as communication, computation and storage resources. The service provider represents its available resources as . is the amount of resources of type available at service provider . The vector represents all available resources at different service providers. Each service provider has a set of native applications that ask for resources. The set of all applications that ask for resources from the set of service providers (coalition of service providers) is given by where i.e., each application originally asks only one service provider for resources. Every player has a request (requirement) matrix .
(6) 
where the row corresponds to the application while columns represent different resources, i.e., is the amount of resource that application requests. Request matrix is an augmentation of all the request matrices (from all the service providers) as given below.
(7) 
where is the amount of resource that application requests.
The resource sharing and allocation framework, based on and , has to make an allocation decision that optimizes the utilities of all the service providers and satisfy user requests as well. The allocation decision is a vector given by that indicates how much of each resource is allocated to application at service provider . The matrix is an augmentation of different matrices , where is the allocation decision at service provider for the applications originally belonging to service provider . is the allocation decision at service provider for applications belonging to service provider , i.e., the original applications of the service provider . Mathematically,
(8) 
where is the amount of resource belonging to player that is allocated to application .
Iiia Optimization Problem
In this section, we first present the resource allocation problem for a single service provider (no resource sharing with other service providers). Then we present the MOO problem for resource sharing and allocation. For a single cloud (not part of a coalition), the allocation decision matrix , where stands for single objective, is given by:
(9) 
The optimization problem is:
(10a)  
s.t.  (10b)  
(10c)  
(10d) 
The goal of this single objective optimization problem for every service provider is to maximize its utility by allocating available resources only to its own applications. captures the satisfaction of resource requests by applications.
Next, we present the resource allocation problem for different service providers that share their resources with each other. In the resource sharing case, each service provider aims to maximize sum of utilities obtained by a) allocating resources to its native applications, i.e., ; b) allocating resources to applications of other service providers, i.e., ; and c) satisfying requests of applications. This leads to:
(11a)  
s.t.  (11b)  
(11c)  
(11d) 
The first constraint in both optimization problems, (10) and (11), indicates that the allocated resources cannot be more than the capacity. The second constraint in (10c) and (11c) indicates, that the allocated resources should not be more than the required resources. The last constraint, (10d) and (11d), indicates that allocation decision cannot be negative.
is the utility that domain receives for sharing its resources with domain . is the weight associated with the utility and is the weight affiliated with all . In this paper, we assume that utility of the service providers is a nondecreasing monotone function. We also assume that a subset of service providers has resource deficit, while a subset has a resource surplus.
IiiB Game Theoretic Solution
As mentioned earlier, we model each cloud service provider as a player in our game to obtain the solution for the MOO problem in (11). Let be the set of players that can play the resource sharing and allocation game. The value of coalition for the game players is given by (12), where is the feasible set.
(12) 
Theorem 1.
Resource allocation and sharing problem (with multiple objectives) for the aforementioned system model can be modeled as a canonical cooperative game with NTU.
Proof.
To prove this, we need to show that the characteristic function of resource sharing and allocation problem satisfies the following two conditions:

Characteristic form of payoff: As the utility function in a resource sharing and allocation problem only relies on the service providers that are part of the coalition, the game or payoff function is of characteristic form.

Superadditivity: For any where , . The proof follows from definition of monotone utilities.
∎
Definition 14 (Convex Games).
A coalition game is said to be convex if and only if for every player , the marginal contribution of the player is nondecreasing with respect to (W.R.T.) set inclusion. Mathematically, for
(13) 
Theorem 2.
Our canonical game is convex.
Proof.
Let us consider two coalition and , where , , and and are the feasible sets for and respectively that it achieves by allocating resources to its own applications. We calculate in Equation (IIIB), where and are the utilities of player in and respectively.
(14) 
The last inequality in (IIIB) follows from the fact that (monotonicity of the utility). ∎
Remark 2.
The core of any convex game is nonempty and large [10].
Remark 3.
Our canonical cooperative game with NTU can be used to obtain the Paretooptimal solutions for the multiobjective optimization problem given in (11).
While the existence of core, i.e., core being nonempty, guarantees the grand coalition is stable, finding a suitable allocation from the core is challenging particularly when the core is large (our game). Algorithm 1 provides an allocation from the core^{2}^{2}2Provided that the aforementioned assumptions hold., by solving optimization problems.
Theorem 3.
The allocation decision obtained using Algorithm 1 lies in the core.
Proof.
To prove the theorem, we need to show that the allocation decision obtained using Algorithm 1: a) is individually rational; b) is group rational; and c) no players have the incentive to leave the grand coalition and form another subcoalition .
Individual Rationality: For each player , the solution obtained using (15) is individual rational due to the constraint in (15b).
Hence the solution obtained as a result of Algorithm 1 is individually rational.
Group Rationality: The value of the grand coalition as per Equation (15a) is the sum of utilities of all players that they achieve from the payoff vector . Hence the allocation obtained from Algorithm 1 is group rational.
Furthermore, due to superadditivity of the game and monotonic nondecreasing nature of the utilities, no subgroup of players have an incentive to form a smaller coalition. Hence Algorithm 1 provides a solution from the core. ∎
In algorithm 1, we first solve the single objective optimization problem (10) for all players . We then solve the problem in (15) that provides the allocation from the core. in (15b) is the payoff a player receives when working alone.
(15a)  
s.t.  
(15b) 
In the next section, we evaluate the performance of our resource sharing and allocation framework.
Iv Experimental Results
We evaluate the performance of proposed resource sharing and allocation framework for a number of settings as shown in Table I. Each player has three different types of resources (), i.e., storage, communication and computation. Without loss of generality, the model can be extended to include other type of resources/parameters. We used linear and sigmoidal utilities (see (16)) for all the players. However, the results hold for any monotone nondecreasing utility.
(16) 
is chosen to be either or to capture the requirements of different applications. The request matrices and the capacity vectors are randomly generated for each setting within a prespecified range^{3}^{3}3The larger the number of applications in our simulation settings, the larger is range for random number generation.. To show the advantage of resource sharing, we allocate larger capacities to certain players that, for improving their utilities, share the available resources with other domains and assist other players in meeting demand. Since all the service providers do not differentiate between their native and applications of other service providers, we set and to .
Setting  Parameters 

1  
2  
3  
4 
The simulations were run in Matlab R2016b on a Corei7 processor with 16 GB RAM. To solve the optimization problems, we used the OPTItoolbox. Below, we provide detailed experimental results.
Iva Verification of gametheoretic properties
In Table II, we present results for a 3player 20application game that verify different game theoretic properties such as individually rationality, group rationality, super additivity and show that the obtained allocation is from the core. The payoff all players receive in the grand coalition, i.e., is at least as good as players , and working alone. This shows that the solution obtained using Algorithm 1 for the grand coalition is individually rational. Similarly, the value of coalition is the sum of payoff all players receive is the value of coalition, hence our solution is group rational. Furthermore, with an increase in the coalition size, value of coalition also increases. Hence, the grand coalition has the largest value, which shows the superadditive nature of the game. Also, no set of players has any incentive to divert from the grand coalition and form a smaller coalition. Hence, the grand coalition is stable and the allocation we obtain using Algorithm 1 is from the core. Similar results were seen for other settings given in Table I. However, we do not include them here due to space constraints.
Coalition  Player 1  Player 2  Player 3  Value of coalition 
{1}  584.40  0.00  0.00  584.40 
{2}  0  90  0  90 
{3}  0  0  90  90 
{1, 2}  584.40  227.15  0  811.55 
{1, 3}  584.40  0  229.84  814.24 
{2, 3}  0  118.91  119.50  238.41 
{1, 2, 3}  584.40  205.11  202.30  991.81 
IvB Efficacy of the resource sharing framework
IvB1 Player utilities
Figure 3 shows the utility of game players in different simulation settings. It is evident from the figure that the utility of players in grand coalition (GC) in all simulation settings either remains as good as working alone (very small number of cases) or drastically improves with our game theoretic resource sharing framework. This shows that cloud service providers can greatly benefit by employing a resource sharing scheme. The average improvement in the utility of players seen in our simulations is almost 207%. However, it is worth mentioning that the improvement greatly depends on parameters such as resource surplus, deficit, and the utilities used.
IvB2 User satisfaction
Satisfying user requests is important for cloud service providers as it is mandated by the SLA. Furthermore, it helps in retaining their users (customers) resulting in longterm profits. Figure 4 shows the average user request satisfaction in different playerapplication settings. For a player game with and applications respectively, player has a resource deficit and is not able to satisfy all of its user’s request when operating alone. Players and have a resource surplus so they can satisfy their users even working alone. When all the three players work in a coalition, the request satisfaction for all three players is 100%. Similarly, for a player game with and applications respectively, we see that players who could not satisfy their applications alone are able to satisfy all their user requests by joining the coalition and “renting” resources from the players that have a resource surplus. It is worth mentioning that resource sharing can drastically improve user satisfaction. However, the extent of improvement depends on the nature of the utility function, capacity and resource requests.
IvB3 Resource utilization
Figure 5 shows the resource utilization at different players in various simulation settings. For a player game with and applications respectively, we see that the resource utilization for player and (players with resource surplus) increases in the grand coalition when compared with working alone. This is because these players rent out their resources to player that may need them resulting in an increased utility. Player with resource deficit had 100% resource utilization rate working alone. However, resource utilization for player in grand coalition either may remain the same or reduce, since its users’ requests may be satisfied by other players. Similar results are seen for a player game with and applications as evident from the figure.
It is evident from our results that the use of resource sharing and allocation is beneficial for cloud service providers. By allowing different cloud service providers to cooperate, the utility of service providers increases and resources are utilized in an optimal manner. All the service providers in the game have the incentive to work together and use their resource capacities in the best possible way.
V Related Work
There have been a number of solutions proposed in literature related to resource availability and resource allocation in cloud computing. Alicherry et al. [11] presented a 2approximation algorithm for network resource allocation in a distributed cloud environment. Ergu et al. [12] proposed a model for resource allocation in cloud computing environment. Tsai et al. [13] proposed an improved differential evolution algorithm for resource allocation in cloud that combines differential evolution algorithm with Taguchi method. Pawar et al. [14] proposed a dynamic resource allocation that considers different SLA parameters and premptable task execution. Wei et al. [15] proposed a noncooperative game theory based approach for resource allocation in the cloud. The objective of the algorithm is to maximize fairness among different users. Duan et al. [16] modeled scheduling in hybrid clouds as a sequential cooperative game. The authors proposed a storage and communication aware algorithm that jointly optimizes the execution time and economic cost of scheduling BagofTasks work flows. Shi et al. [17] proposed a Shapley value based mechanism for ondemand bandwidth allocation between data centers. They focus on network bandwidth and do not take computing and storage resources into account. Shi et al. [1] also proposed an online auction mechanism for dynamically providing virtual clusters in geodistributed clouds. Zafari et al. [18] modeled resource sharing among mobile edge clouds as a canonical game with transferable utility (TU).
However, our work in this paper differed from [1, 17, 15, 11, 16]. We considered the multiobjective nature of the resource sharing problem and allowed different cloud service providers to share resources and improve their utilities, while satisfying the requests of different users. Our work is closest to [18]. However, authors in [18] considered the case where each service provider first allocates resources to its own applications and shares the remaining resources with the applications of other service providers. They only considered the utility of the service provider and did not consider the user satisfaction. Our framework guarantees Pareto optimality like [18], however, it is more generic than [18] as TU games can be modeled as a special case of NTU [9].
Vi Conclusions
In this paper, we proposed a cooperative game theory based framework for resource sharing and allocation among cloud service providers. We showed that for a monotonic nondecreasing utility, resource sharing among multiple cloud service providers can be modeled as a canonical game with nontransferable utility. We prove that the game is convex, hence the core of the game is nonempty. We proposed an algorithm that provides allocation decision from the core. Simulation results showed that our proposed framework improves utility of cloud service providers. Furthermore, request satisfaction of users also improved.
References
 [1] W. Shi, C. Wu, and Z. Li, “An Online Auction Mechanism for Dynamic Virtual Cluster Provisioning in GeoDistributed Clouds,” IEEE Transactions on Parallel and Distributed Systems, vol. 28, no. 3, pp. 677–688, 2017.
 [2] Z. Xiao, W. Song, Q. Chen et al., “Dynamic resource allocation using virtual machines for cloud computing environment.” IEEE Trans. Parallel Distrib. Syst., vol. 24, no. 6, pp. 1107–1117, 2013.
 [3] W.Y. Lin, G.Y. Lin, and H.Y. Wei, “Dynamic Auction Mechanism for Cloud Resource Allocation,” in Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing. IEEE Computer Society, 2010, pp. 591–592.

[4]
A. Beloglazov, J. Abawajy, and R. Buyya, “EnergyAware Resource Allocation Heuristics for Efficient Management of Data Centers for Cloud Computing,”
Future generation computer systems, vol. 28, no. 5, pp. 755–768, 2012.  [5] V. MayerSchönberger and K. Cukier, “Big Data: A Revolution That Will Transform How We Live, Work, and Think,” 2013.
 [6] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A Survey,” Computer networks, vol. 54, no. 15, pp. 2787–2805, 2010.
 [7] J.H. Cho, Y. Wang, R. Chen, K. S. Chan, and A. Swami, “A Survey on Modeling and Optimizing MultiObjective Systems,” IEEE Communications Surveys & Tutorials, vol. 19, no. 3, pp. 1867–1901, 2017.
 [8] Z. Han, Game Theory in Wireless and Communication Networks: Theory, Models, and Applications. Cambridge University Press, 2012.
 [9] R. B. Myerson, Game theory. Harvard university press, 2013.
 [10] W. W. Sharkey, “Cooperative Games with Large Cores,” International Journal of Game Theory, vol. 11, no. 34, pp. 175–182, 1982.
 [11] M. Alicherry and T. Lakshman, “Network Aware Resource Allocation in Distributed Clouds,” in Infocom, 2012 proceedings IEEE. IEEE, 2012, pp. 963–971.
 [12] D. Ergu, G. Kou, Y. Peng, Y. Shi, and Y. Shi, “The Analytic Hierarchy Process: Task Scheduling and Resource Allocation in Cloud Computing Environment,” The Journal of Supercomputing, vol. 64, no. 3, pp. 835–848, 2013.
 [13] J.T. Tsai, J.C. Fang, and J.H. Chou, “Optimized Task Scheduling and Resource Allocation on Cloud Computing Environment using Improved Differential Evolution Algorithm,” Computers & Operations Research, vol. 40, no. 12, pp. 3045–3055, 2013.
 [14] C. S. Pawar and R. B. Wagh, “Priority Based Dynamic Resource Allocation in Cloud Computing with Modified Waiting Queue,” in Intelligent Systems and Signal Processing (ISSP), 2013 International Conference on. IEEE, 2013, pp. 311–316.
 [15] G. Wei, A. V. Vasilakos, Y. Zheng, and N. Xiong, “A GameTheoretic Method of Fair Resource Allocation for Cloud Computing Services,” The journal of supercomputing, vol. 54, no. 2, pp. 252–269, 2010.
 [16] R. Duan, R. Prodan, and X. Li, “MultiObjective Game Theoretic Scheduling of BagofTasks Workflows on Hybrid Clouds,” IEEE Transactions on Cloud Computing, vol. 2, no. 1, pp. 29–42, 2014.
 [17] W. Shi, C. Wu, and Z. Li, “A ShapleyValue Mechanism for Bandwidth on Demand between Datacenters,” IEEE Transactions on Cloud Computing, vol. 6, no. 1, pp. 19–32, 2018.
 [18] F. Zafari, J. Li, K. K. Leung, D. Towsley, and A. Swami, “A GameTheoretic Approach to MultiObjective Resource Sharing and Allocation in Mobile Edge,” in Proceedings of the 2018 on Technologies for the Wireless Edge Workshop. ACM, 2018, pp. 9–13.