I Introduction
As a new resource provisioning and computing model, cloud computing makes it easier and more convenient for users to obtain configurable resources demand in an online manner [52][43]. It not only lowers the online deployment threshold and management complexity but also brings the benefits of reliability and lowrisk [32][45]. With the gradual maturity of cloud computing technology, the demand for cloud service by the small and large scale industries are continuously rising, which lead to the expansion of the data center together [8]. The choice of virtual machine placement (VMP) strategy is vital to lower operational cost for cloud service providers.
Virtual machines (VMs) are created according to the hardware computing resources demand of tenants, i.e., CPU, RAM and disk. They independently run an operating system and some applications [16]. The virtualization technology lies the key to the success of cloud computing, which allows multiple VMs to run on the same physical server (PS) simultaneously without mutual interruption on each other [31]. The VMP problems can be given as to seek an optimal solution for placing VMs onto PSs [6][51]. Since the specified VMP problems can refer to NPHard problems, it poses challenges for the researchers in finding an optimal solution [24][36].
Up till now, a series of VMP strategies, including heuristic and mathematical methods, have been investigated from different perspectives. According to the difference of target optimization objective, existing researches can be loosely classified as follows: i) target for energy efficient optimization, e.g., an energyefficient adaptive resource scheduler for networked fog centers (NetFCs)
[39], a minimum energy VM scheduling algorithm (MinES) [11], a holistic virtual machine scheduling algorithm (GRANITE) [25], an energyefficient knee pointdriven evolutionary algorithm (EEKnEA) [50]; ii) target for network traffic optimization, e.g., a reliable placement based on the multioptimization with trafficaware algorithm (RPMOTA) [30], a trafficaware VM optimization (TAVO) scheme on nonuniform memory access (NUMA) systems [10], a multiobjective ACS algorithm (ACSBVMP) [35]; iii) target for resource allocation optimization, e.g., a correlationaware VMP scheme [9], a layered progressive resource allocation algorithm [21], an energyaware resource allocation method (EnReal) [47], a twotiered ondemand resource allocation mechanism [40]. Although a serious of methods have been successfully developed to address the VMP problems in cloud computing, most of them are carried out on smallscale VMs simulation experiments.Evolutionary algorithms (EAs) are classified as one of the populationbased heuristic methods that simulating the evolutionary process of the population with optimization characteristics [4]. Owing to the effective global populationbased search, EAs have been shown their problemsolving in VMP problems [46][2][28]. But the limitation of them is that they often require a large number of fitness evaluations and high time computation to obtain promising solution [27][22]. As the number of VMs increases in the data center, the VMP problem becomes more complicated and computational expensive. Conventional EAs, which are typically designed to solve a single optimization task, have few practical values on solving the largescale virtual machine placement (LVMP) problem in the actual cloud environment.
Noted that many realworld optimization tasks tend to be potentially correlated [15][14], and some useful knowledge learning from one task could be applied to learn another related task efficiently [5][17]. Motivated by multitask learning [7][34], the multifactorial optimization (MFO) technology was introduced to evolve multiple tasks simultaneously by sharing good traits among them in the field of evolutionary computing [19][18]. Principally, MFO offers the ability to fully exploit the implicit parallelism of populationbased search during the evolutionary process [53][23].
This paper aims to develop a new EAbased MFO method to complete the placement of VMs onto PSs from the optimization target of resource allocation. The core idea of this paper is depicted in Fig. 1 and described as follow. When optimizing the LVMP problem that considered the populationbased methods, the population have to contain the information of all VMs in the data center. Nevertheless, it is worth to note that the VMs do not interfere with each other in the data center. Based on this observation, a crucial inspiration is that a single representation of individual in population is actually not desired to preserve the information of all VMs in the data centers. Further, it means that the LVMP problem could be potentially broken down into multiple smallscale virtual machine placement (SVMP) problems, which could be seen as the form of MFO problem. Therefore, the MFO technology is naturally employed to solve multiple SVMP problems within single population simultaneously. As shown in Fig. 1, after the iteration process, the solution of the LVMP problem can be gained from the obtained solutions of the established MFO problem. Based on the description above, the main contributions of this paper are summarized as follows:
i). The deployment cost based VMP problem is formulated as the form of the MFO problem in heterogeneous environments of the largescale data center.
ii). A multifactorial evolutionary algorithms (MFEA) coupled with greedybased allocation operator is proposed to solve the established MFO problem.
iii). A remigration and merge operator is designed to provide the integrated solution of LVMP problem from the solutions of MFO problem.
The rest of this paper is organized as follows. The related work of EAbased VMP methods and the foundation knowledge of MFEA are reviewed in Section II. Section III describes the formulation of VMP problem and the overall framework of the proposed method is provided in Section IV. Section V discuss the simulation experiments and analysis of our proposed method. Finally, Section VI deals with conclusion and future work.
Ii Background
Iia EAbased VMP Methods
Different from the mathematical optimization methods, EAbased methods find the optimal solution through population iteration search without mathematical analysis of the target objective function. In recent years, a series of EAbased methods, such as particle swarm optimization (PSO), genetic algorithm (GA), ant colony system (ACS) and so on, have been developed to address various VMP problems in cloud computing, which are reviewed as follows:
i) PSObased VMP methods. A.P. Xiong et al. [46] designed energy efficient VMs allocation problem using PSO technology. In their study, the fitness function of PSO is defined as the total Euclidean distance to determine the optimal point between resource utilization and energy consumption. But its limitation of this work is that authors considered single type of VMs. P. Aruna et al. [3] explored a PSO algorithm for the VM provisioning to make the cloud data centers as power efficient, which involves the discussion of the power model for the servers and the implement of the proposed power aware PSO algorithm for the VM provisioning. Recently, another modified binary PSO is proposed by A. Tripathi et al. [42] and applied for solving the VMP problem. It aims to addresses two important Cloud aspects, e.g. efficient energy usage and effective resource utilization.
ii) GAbased VMP methods. David Wilcox et al. [44] formally defined multicapacity bin packing, which was the generalization of conventional bin packing. And they also develop an reordering grouping genetic algorithm (RGGA) to assign VMs to servers. A. S. Alashaikh et al. [2] adopted the notion of ceteris paribus as an interpretation for the decision maker’s preferences and incorporated it in a constrained multiobjective problem known as VMP problems. They proposed a variant of the NSGAII algorithm that promotes ceteris paribus preferred solutions and evaluate its applicability. X. Xu et al. [47] built a manyobjective VMP optimization model target to minimize energy consumption and maximize load balance, resource utilization, and robustness. They also proposed an energyefficient knee pointdriven evolutionary algorithm (EEKnEA) to address their optimization model.
iii) ACSbased VMP methods. An early study using ACS for VMP problems was introduced by E. Feller et al. [13]. In their study, the VMP problem was formulated as a static multiple dimensional bin packing problem, which the optimization goal was to minimize the number of cloud servers for support current load. To solve the formulated optimization problem, an ACS algorithm was also developed in [13] coupled with a maxmin updated method. Nevertheless, this method installs VMs in PSs only based on a single resource. Another study adopting ACS for VMP problems was introduced by X. Liu et al. [29], which consolidates VMs according to multiple resources. In their approach, a different approach was designed to update the pheromones between two pairs of VMs to measure their need for PM in ACS. The number of PSs is the same as the number of the VMs in the first generation and it was decreased as the evolutionary proceed. This work on ACS for VMP problem was improved later by X. Liu et al. [28], which involves an proposed order exchange and migration (OEM) local search techniques to transform an infeasible solution into a feasible solution.
In addition to the above three categories, there are other heuristic algorithms for solving VMP problems. For example, N. K. Sharma et al. [38] introduced approach of GA and PSO referred to as HGAPSO for the placement policy of VMs to PSs. X. Li et al. [26] proposed discrete firefly algorithm to solve VMP problems, which takes firefly’s location as the placement result and brightness as the objective value.
IiB MultiFactorial Evolutionary Algorithm (MFEA)
The multifactorial evolutionary algorithm (MFEA) is able to evolve two or more tasks simultaneously and accelerate the convergence of each task [19][33]. The unified representation block is built in MFEA to achieve knowledge transfer among tasks, which include the encoding and decoding operation. The encoding operation is called to build an unified express space denoted as . The individual in includes the genetic material of all tasks. The dimension of can be defined as = max{}, where is the dimension of the th task and is the number of tasks, = 1, 2, …, . Reversely, the decoding step can decipher into taskspecific solutions with the th solution being = (1: ), where (1: ) retrieves the first dimensions of . Some foundation definition of each individual are described as follow:
Factorial Cost: The objective function value of individual on task is defined as the factorial cost of . The factorial costs of on the other tasks are set to infinity.
Factorial Rank: The factorial rank of individual is defined as the rank of on the th task considering all individuals in .
Skill Factor: The skill factor of individual indicates the task on which performs the most effectively, i.e., , where k = 1, 2, …, K. For the sake of simplicity, we say an individual is specific to a task if the task is the skill factor of .
Scalar Fitness: The scalar fitness of individual is denoted as the reciprocal of the corresponding factorial rank on task , i.e., . A greater scalar fitness value means that
can survive to the next generation at a higher probability.
Further, the pseudo code of MFEA is presented in Algorithm 1, which can be summarized as follow.
In the beginning, the initial operations, e.g., randomly generating N individuals in a unified express space Y and assigning the skill factor to each individual in the initial population, are called to form a population. Then, the crossover [12] and polynomial mutation [20] operations are employed to reproduce offspring according to the assortative mating and vertical cultural transmission mechanism. Notably, the assortative mating and vertical cultural transmission are the characteristic and essential components of MFEA, which allow individuals from different tasks to share genetic information with each other. Specifically, the assortative mating mechanism enables the individuals from different tasks to mate with each other at a certain probability, namely random matting probability (rmp). The vertical cultural transmission mechanism randomly assigns skill factors to the offspring. It means that the offspring specific to one task may be switched to another directly, leading to a complex optimization task that may acquire superior solutions by learning from other tasks. More details of assortative mating and vertical cultural transmission are available in [19]. Afterward, the factorial cost, factorial rank, and scalar fitness of each offspring individual are updated. Finally, the elitebased environment selection operator is employed to form the next generation population.
In recent years, MFEA and its variants have been successfully applied to various realworld optimization problems. In particular, L. Zhou et al. [54] proposed the MFEA equipped with a permutationbased unified representation and splitbased decoding operator to solve the NPhard capacitated vehicle routing problems. H. ThiThanh Binh et al. [41] proposed a modified MFEA for cluster shortest path tree problems (CSTP), together with novel genetic operators, e.g., population initialization, crossover, and mutation operators. Furthermore, a novel decoding scheme for deriving factorial solutions from the unified representation in MFEA, which is the critical factor to the performance of any variant of the MFEA, is also introduced in [41]. C. Yang et al. [48] extend their method TMOMFEA proposed in [49] to solve operational indices optimization, which involves a formulated multiobjective multifactorial operational indices optimization problem. In their proposed optimization model, the most accurate task is considered to be the original functions of the solved problem, while the remained models are the helper tasks to accelerate the optimization of the most accurate task.
Iii The formulation of VMP Problem
The VMP problems can be seen as linear programming (LP) problems, which usually consider hardware resource constraints of PSs such as CPU, RAM and disk. As the industry’s demand for VMs continues to grow, the singletask EAbased solvers have suffered from some limitations when addressing a LVMP problem. Most of them require more optimization time to complete the allocation of VMs onto PSs in the largescale data center, which resulted in the poor scalability. In this paper, the LVMP problem is reformulated in the form of the MFO problem, which is described as follows. The used variables and their definitions are summarized in Table
I.(1) 
where represents a SVMP optimization task that decomposed from the LVMP problem, which is described as follow. And is the solution of th optimization task , = 1, 2 … . is the total number of SVMP optimization tasks.
The main differences between the established MFO problem and other MFO problems lie in two aspects presented as follows. On the one hand, the existing MFO problems aim to solve multiple optimization tasks, and the obtained solution set are corresponding to all tasks. Conversely, the purpose of established MFO problems is to address the single optimization problems. On the other hand, the number of optimization tasks is changed dynamically. It means that varies with the size of the VMs in the data center.
Assume that there are VMs and PSs in the data center. Given VMs assigned to th VMP optimization task, the number of VMP optimization tasks is defined as follows:
(2) 
Considering that the relationship between and is not always divisible, the number of VMs assigned to the th VMP optimization task , e.g., = , is reallocated as follows:
(3) 
where the symbol is the mod operation. Owing to the organic composition of different types of PSs, the data center could be operated with high utilization of PS clusters and achieved energy efficiency. However, the existing VMP researches generally designed the optimization model to lower the number of activated PSs or improve the utilization of PSs cluster, which ignore the heterogeneity between PSs. In the actual cloud environment, when a PS is activated, it brings deployment costs that depends on its configuration. It means that the deployment costs can be naturally used to guide which type of PS to activate for improving the utilization of PSs cluster. According to the description above, this paper proposes a deployment cost based optimization model to solve VMP problems in heterogeneous environment.
Assume that PSs are categorized in to different types according to the specific configuration, where 1. The number of each type of PSs is denoted as , where . Noted that PSs is also assigned to the th task, which could be given as:
(4) 
The assignment tag of VMs in th task is denoted as , which is a matrix consisted of 0/1 variables. If the in th task is placed on , then the element of =1, otherwise =0. Similarly, the activated tag of PSs in th task is denoted as , which is a vector consisted of 0/1 variables. If there are greater than or equal to one VM placed in , then the element of = 1, otherwise = 0. The element and p are formulated as follows:
(5) 
(6) 
In this work, three typical computing resources (CPU, RAM and disk) in cloud computing are considered as constraint options. Based on the above description, the deployment cost based SVMP optimization task for the established MFO problem can be given as follows:
(7) 
where , and represent the computing source requirement CPU, RAM and disk of the th VM in th optimization task, respectively.
Symbols  Definition 
Number of VMs in data center  
Number of PSs in data center  
Number of PSs types in data center  
Number of VMP optimization task  
The th VMP optimization task  
Solution of th VMP optimization task  
Number of VMs assign to each VMP optimization task  
Number of PSs assign to th VMP optimization task  
Number of th type PS in data center  
Binary variable of VM placement  
The th VM in th VMP optimization task  
CPU requirement of VM  
RAM requirement of VM  
Disk requirement of VM  
Binary variable of PS status  
The th PS in data center  
Deployment cost of the th PS  
CPU capacity of PS  
RAM capacity of PS  
Disk capacity of PS 
Iv Our Proposed Method
Iva The Overall Framework
The overall framework of our proposed method is described in Algorithm 2.
Line 1 to 2 represent the pretreatment process of the input VMs list. The input VMs list is randomly assigned to H optimization tasks. Line 3 is the process of building unified express space for H VMP optimization tasks, which is presented in subsection IV.B in detail. Lines 4 to 6 describe a series of initialization operator for generating the population in unified express space. Lines 8 to 9 are the process of offspring reproduction, which is similar to the basic MFEA. The crossover and mutation operators used in assortative matting are described detailedly in subsection IV.D. After that, the generated offspring population is assigned the skill factor according to the vertical cultural transmission. Line 10 is the proposed greedybased allocation operator, which is described in subsection IV.C. Line 11 is the evaluation of offspring population and lines 12 to 13 represent the updated the scalar fitness for both the offspring and parent population. Line 14 forms the population to survive into the next generation based on the elitebased environment selection. Lines 7 to 15 are executed repeatedly until the termination condition is satisfied. Line 16 is the proposed remigration and merge operator, which is called to obtained the placement solution of the input VMs list.
IvB Building Unified Express Space
In this subsection, the procedure of building the unified express space is described as follows.
To achieve the implicit knowledge transfer among H VMP optimization tasks, it is essential to construct a unified genetic express space among them. The individuals, which are generated in unified express space, contain all the information of VMs for each task. Note that the number of the same VM types is not equal in each VMP optimization task due to the random assignment process. Different types of VM refer to the different configuration (such as CPU, RAM, or disk). For each different types of VMs, take the maximum number of VMs in all tasks as the number of this types of VMs in unified express space.
Fig. 2 illustrates an example of establishing the unified express space. Suppose that there are five types of VMs in the data center. Their type ID is denoted as type1, type2,…, type5, respectively. And all the VMs are assigned to three VMP optimization tasks randomly. Take the type2 as an example, the number of type2 assigned to the first VMP optimization task is 1, the number of type2 assigned to the second VMP optimization task is 2, and the number of type2 assigned to the third VMP optimization task is set to be 2. As a result, the number of type2 is 2 in unified express space. Other types of VMs do the same operation. After the transform process above, the individuals generated in unified express space retain the information of each VMP optimization task and it could be decoded into all tasks.
The representation block of individual in unified express space is shown in Fig. 3. In a individual, if the PS contains VMs, it is assigned 1, otherwise 0. The PSs list not only consists of binary number, but records the information of VMs with which it loaded. In Fig. 3, for example, the symbol of first PS is assigned 1, which indicates that the first PS is activated and installs the type1 and type3 VMs.
Compared with the singlefactorial solver, the representation block of individual in our method is more effective. Take Fig. 2 as an example, suppose there are 23 VMs in the data center. An possible individual in the unified express space is shown in Fig. 2, it can be seen that each individual generated in unified express space only needs to contain the information of 10 VMs. And the representation block of individual in singlefactorial solver is consisted of the information of 23 VMs. The individual with fewer information of VMs means that it can complete the placement of VMs onto PSs in a faster time obviously.
IvC Greedybased Allocation Operator
In this subsection, the proposed greedybased allocation operator is described as follows.
The VMs list of individual in the unified express space need to be decoded into its corresponding task before the greedybased allocation operator is performed. The decoding operation takes the first n VMs that meet the types and number requirement of VMs assigned to a task to generate the VMs list. Then the greedybased allocation operator is performed on the generated VMs list. An example of the decoding operation is depicted in Fig. 4. As shown in Fig. 4, a possible VMs list of individual in unified express space is {1, 2, 3, 5, 4, 5, 3, 2, 5, 4}. After the decoding operation, the generated VMs list for one of the specific task is {1, 2, 3, 5, 4, 3, 4}.
The pseudo code of greedybased allocation operator is summarized in Algorithm 3. The data center has a variety of different configuration PSs. For each type PS, we select one to load the generated VMs list until any specific resource (CPU, RAM, or disk) of the selected PS is exhausted or there are no VMs can be loaded in the selected PS. Then we select the PS with the highest comprehensive utilization rate and erase the VMs it loaded from the generated VMs list. The definition of comprehensive utilization rate U of a single PS is below:
(8) 
where the parameters , , and are the weight coefficient. To ensure the importance of three selected hardware resource is the same, we set , and as the same weight to guarantee the fairness of them, i.e., = = = 1/3. The utilization of CPU , the utilization of RAM and the utilization of disk are given as follows:
(9) 
(10) 
(11) 
where , and are the CPU, RAM and disk resource capacity of th PS, respectively.
IvD Crossover and Mutation Operator
The offspring population is reproduced by assortative mating. The crossover and mutation operator used in assortative mating are described in detail as follow.
Crossover operator: The crossover operator aims to reproduce offspring population theoretically with good traits from both parents and hopefully better fitness [44]. In this paper, the crossover operator is implemented in an modified exon shuffling approach similar to the one described by [37]. This approach combines all the activated PSs from randomly selected parents and sorts the PSs by comprehensive utilization, which is calculated based on Eq. (8). The more full PSs are at the front of the list, while the less full PSs are at the end. The crossover operator systematically picks the more full PSs and keeps the VMs that have loaded intact. If the picked PS contains any VMs that have been installed in other PS, the picked PS is discarded and set to be idle status. The remaining VMs that have not been placed in any PSs are randomly disrupted and inserted to the end of the generated VM list. This crossover approach preserves as many VM lists as possible in more fullfilled PSs. It also ensures that the generated candidate solutions are all feasible solutions and it is avail to speed up the running time of the algorithm.
Mutation operator: After the crossover operator, each individual is allowed to having their candidate solution modified slightly at a certain probability. The mutation operator aims to facilitate the population to escape from local optima. In this paper, it is realized by randomly swapping the position of two VMs in the VMs list of individual.
IvE Fitness Evaluation
After the individual has finished the placement process, we evaluate it fitness according to the status of PSs (activated or idle). The fitness evaluation function is define as the deployment cost of activated PSs list of individual in th task, which is given as follows:
(12) 
where is the deployment cost of th type PS. After the fitness evaluation process, the scalar fitness of the individual in population that consists of the parent and offspring is updated. Then, in the elitebased environment selection operator, the individuals with smaller scalar fitness are survived to the next generation. In particular, the number of individuals from different tasks survived to the next generation is equal.
IvF Remigration and Merge operator
Although the obtained solutions of each tasks is feasible, they are not the solutions of the original input VMs list. The designed remigration and merge operator is called to offer the solution of the original input LVMP problems from the obtained solutions of MFO problem, which the pseudo code is presented in Algorithm 4.
For the fittest solution in each VMP optimization task, the remigration operator is employed to the PS list that are not fully utilized by computing resources, e.g., the available computing resources are left behind. Specifically, the VMs that loaded in the PS which have the remaining compute resource are repopped, which the new VMs list is generated to be refilled by the greedybased allocation operator. Finally, the merge operator is executed to combine the activated PSs among all the fittest solution and form the placement scheme of the LVMP problem.
IvG Time Complexity Analysis
Besides the capability of finding solutions of high quality, time complexity is a significant issue for an optimization algorithm. The time complexity of our proposed method is mainly based on GA. Assume the maximum iteration is “I”, the number of PSs is “M”, the number of PSs types is “L”, the number of VMs is “V”, and the generation size is “G”. The time complexity of our method is mainly depended on the offspring reproduction and greedybased allocation operator. Further, as seen in Algorithm 3, the time complexity of proposed greedybased allocation operator is (iteration generation size the number of VMs the number of PSs types the number of VMs), which is (I G V L V). And the the offspring reproduction require (I G M V) time complexity. Since the number of VMs is larger than the number of PSs, based on the analysis above, the time complexity of our method is (I G V V L).
V Experiments
Simulation Experimental tests are carried out in this section to assess the performance of our proposed. All the compared algorithms have been implemented in C++, and ran on a PC with a Pentium Dual CPU i7 and 8.0 GB RAM.
Va Test Data Set Introduction
Type  CPU  RAM  Disk  Cost  
General 





LargeRAM 





HighPerformance 




In this work, the resources characterize of VMs and PSs are downloaded from the Huawei cloud website^{1}^{1}1https://activity.huaweicloud.com/promotion/index.html. Three different types of PSs are adopted to build heterogeneous environments, which are respectively denoted as General, LargeRAM and HightPerformance. The configurations of PSs are shown in Table II. Further, these three different types of PSs have different bias characteristics. LargeRAM has a larger memory, preferring to place the VM with greater RAM demand. HightPerformance has more CPU cores and is suitable for the VMs that required high computing power. The resources characterized of General is between the above two. Besides, we use 100 different types of VMs for the creation of largescale test data sets, and the configuration of VMs are shown in Table III
. The ratio of CPU and RAM of each VM comes from the Huawei cloud website. The configuration of disk is set to 100500 (G) and it is randomly generated at 100 (G) intervals, which is considered as bottleneck resource in this experiment. The total ratio of the requirement of CPU, RAM, and disk is approximately 6:9:10 on each test data set, which proves the resource of disk is served as a bottleneck resource in this experiment. In conclusion, taking the data set with the size of 5,000 VMs as an example, it is composed of 5,000 VMs which randomly produced from 100 different types of VMs with the discrete uniform distribution. Point out that the optimal solution to the test data sets are not clear due to the random creation process, which could be served as blackbox testing problems.
Type  CPU  RAM  disk  Type  CPU  RAM  disk  Type  CPU  RAM  disk 
(core)  (G)  (G)  (core)  (G)  (G)  (core)  (G)  (G)  
type1  1  1  100  type36  2  16  100  type71  8  32  100 
type2  1  1  200  type37  2  16  200  type72  8  32  200 
type3  1  1  300  type38  2  16  300  type73  8  32  300 
type4  1  1  400  type39  2  16  400  type74  8  32  400 
type5  1  1  500  type40  2  16  500  type75  8  32  500 
type6  1  2  100  type41  4  4  100  type76  8  64  100 
type7  1  2  200  type42  4  4  200  type77  8  64  200 
type8  1  2  300  type43  4  4  300  type78  8  64  300 
type9  1  2  400  type44  4  4  400  type79  8  64  400 
type10  1  2  500  type45  4  4  500  type80  8  64  500 
type11  1  4  100  type46  4  8  100  type81  16  16  100 
type12  1  4  200  type47  4  8  200  type82  16  16  200 
type13  1  4  300  type48  4  8  300  type83  16  16  300 
type14  1  4  400  type49  4  8  400  type84  16  16  400 
type15  1  4  500  type50  4  8  500  type85  16  16  500 
type16  1  8  100  type51  4  16  100  type86  16  32  100 
type17  1  8  200  type52  4  16  200  type87  16  32  200 
type18  1  8  300  type53  4  16  300  type88  16  32  300 
type19  1  8  400  type54  4  16  400  type89  16  32  400 
type20  1  8  500  type55  4  16  500  type90  16  32  500 
type21  2  2  100  type56  4  32  100  type91  16  64  100 
type22  2  2  200  type57  4  32  200  type92  16  64  200 
type23  2  2  300  type58  4  32  300  type93  16  64  300 
type24  2  2  400  type59  4  32  400  type94  16  64  400 
type25  2  2  500  type60  4  32  500  type95  16  64  500 
type26  2  4  100  type61  8  8  100  type96  16  128  100 
type27  2  4  200  type62  8  8  200  type97  16  128  200 
type28  2  4  300  type63  8  8  300  type98  16  128  300 
type29  2  4  400  type64  8  8  400  type99  16  128  400 
type30  2  4  500  type65  8  8  500  type100  16  128  500 
type31  2  8  100  type66  8  16  100  
type32  2  8  200  type67  8  16  200  
type33  2  8  300  type68  8  16  300  
type34  2  8  400  type69  8  16  400  
type35  2  8  500  type70  8  16  500 
VB Compared Algorithms
In this experiment, the effectiveness of the proposed method is validated in comparison with other four different types of stateofart heuristic algorithms, namely HGAPSO [38], OEMACS [28], RGGA [44], and firstfit decreasing (FFD) [1]. OEMACS construct the assignment of VMs by artificial ants based on the proposed global pheromone updating approach. At the same time, the proposed order exchange and migration (OEM) technique in OEMACS is able to turn the infeasible solutions into feasible solutions, which aims to improve the quality of the solution. HGAPSO proposed a hybrid method of genetic algorithm (GA) and particle swarm optimization (PSO) to complete the allocation of VMs to PSs. The RGGA used the approach of exon shuffling to produce offspring. It sorts the PSs list by utilization rate in descending order and systematically picks the more full PSs to keep them intact. Finally, the FFD algorithm arranged the VMs list by first considering CPU requirement, then RAM requirement, and lastly disk requirement in descending order. And it installs the VMs in the first PS with adequate computing resource in the PSs list.
VC Experiment Setup
In this experiment, the parameters setting of the proposed method are the random mating probability (rmp) and the number of VMs assigned to th VMP optimization task , which rmp = 0.3 and = 200. The above parameters settings are derived from the sensitivity test experiment, which is presented in subsection V.E. The number of maximal iterations is set to 50, the number of individuals in per task is set to = 5, the total population size is the task number multiplied by , that is, . In addition, the parameters setting of compared algorithm HGAPSO, OEMACS and RGGA set consistently with their original references, with their population size are 10, 5 and 75, and the maximal iteration are 50, 50 and 100, respectively. To conclude, the maximal fitness evaluation of HGAPSO, OEMACS and RGGA are 500, 250 and 7500, respectively. And the fitness evaluation of our method depends on the number of tasks, which is . The experimental results of all algorithms are obtained over 30 independent runs on each instance.
Numerous performance indicators are used to validate the effectiveness of the proposed method comprehensively. They are the running time of each compared algorithm, the number of activated PSs, the comprehensive utilization rate of activated PS cluster, and the total deployment cost. Node that the comprehensive utilization rate of activated PS cluster is calculated by the Eq. (8). Besides these, the utilization of CPU, the utilization of RAM and the utilization of disk are also adopted as performance indicators in this experiment, which is calculated by the Eq. (9), (10) and (11), respectively.
VD Results and Analysis
Data Set NO.  Size  Our Method  HGAPSO  OEMACS  RGGA  FFD  
Time  Util  Num  Cost  Time  Util  Num  Cost  Time  Util  Num  Cost  Time  Util  Num  Cost  Util  Num  Cost  
DS1  5000  0.74s  85.47%  534.26  2316.15 
















DS2  10000  1.48s  85.19%  1056.01  4578.87 
















DS3  15000  2.27s  84.93%  1595.60  6918.30 
















DS4  20000  3.02s  85.27%  2114.13  9174.47 
















DS5  25000  3.81s  84.99%  2661.26  11540.25 
















DS6  30000  4.65s  85.31%  3211.40  13926.30 
















DS7  35000  5.42s  85.11%  3693.43  16026.43 
















DS8  40000  6.25s  85.63%  4281.73  18578.08 
















DS9  45000  6.94s  85.42%  4759.33  20803.48 
















DS10  50000  7.66s  85.01%  5215.50  22620.94 















The experiments results, in terms of overall the performance indicators mentioned in subsection V.C, are obtained by all compared algorithms in 30 independent operations and tabulated in Table IV. The superior average experimental results are represented by bold. Since OEMACS has reached 19514.45s running time in test data set DS7, which has few practical application value. The experimental effect of OEMACS in test data sets DS8DS10 are not given in Table IV. The running time of FFD is only few micro seconds in all test data set, so it is not listed.
It can be seen from Table IV that on all the test data sets, the proposed method achieves significant superior running time compared to other heuristic algorithms. Explicitly, on the test data set DS7, although the fitness evaluation number of our proposed method is 12250, it completes the task of positioning and optimizing the VMs list used only 5.42s, which proves that the proposed method has favorable realtime capability and expansibility. The reason is that the representation of individual in our method is more effective than that of other heuristic algorithms, which result in dramatically shorten the running time. Compared with the other two modified genetic algorithms HGAPSO and RGGA, they run 79.76s and 228.03s on the test data set DS7, respectively. It must be pointed out that HGAPSO can achieve less running time than RGGA due to the original parameter setting of HGAPSO, in which the number of fitness evaluation is much less than that of RGGA. OEMACS need 19514.45s to complete the assignment process on the DS7, which is the most longest running time in all compared algorithms. Further, it can be seen from tendency Fig. 5 of the running time increase is that the proposed method adds only about 1.0s to each additional 5000 VMs list, while other algorithms increase the larger steps. Particularly the increase in running time of OEMACS is the most obvious. The experimental result in terms of the running time proves that the proposed method is more suitable for solving the VMP problem in the largescale data center.
In terms of the average comprehensive utilization rate of activated PSs cluster, our method has achieved the highest comprehensive utilization rate among all the compared algorithms, which shows that it can effectively maximize the utilization rate of the activated PSs cluster. In a deeper matter, Fig. 6 shows the utilization rate of CPU, RAM, and disk on the DS7 of all compared algorithms, respectively. As shown in Fig. 6, compared to RGGA, the proposed method obtains the superior utilization rate on CPU and RAM, and comparable utilization rate on disk, which is considered as bottleneck resource in this experiment. And the proposed method shows higher utilization rate on CPU, RAM and disk in the comparison of HGAPSO, OEMACS and FFD. It shows that our method could effectively respond to LVMP problems of which has the bottleneck resource characteristics.
In addition, the average activated number of PSs cluster by the proposed method is less than other compared heuristic algorithm on all test data sets. For example, the average activated number of PSs in the proposed method is 3693.43 on the DS7, which HGAPSO, OEMACS and RGGA are 4585.46, 4571.50 and 4169.03 respectively. The results proves that our method can effectively reduce the number of activated PSs, accordingly achieve the purpose of reducing energy consumption.
Finally, compared with other algorithms, it can be seen from Table IV that the proposed method obtains lower deployment cost than other compared algorithm. This is due to the fact that each VMP optimization task is modeled as an optimization goal with reduced deployment costs in our proposed method. The advantage of cost based model is its ability to lower the deployment cost of data center and deal with VMP problems in heterogeneous environments. Moreover, since each VMP optimization task has the same optimization goals, which means that there are some higher similarity between them, it can lead to effective communication and obtain better optimization results.
VE Parametric Analysis
In this experiment, the susceptibility of two parameters in the proposed method is tested on the DS1 independently, e.g. random mating probability rmp and the number of VMs assigned to each task . The situations on the other test data sets are similar.
To analysis the susceptibility of rmp, we set its value from 0.1 to 0.9 with a step length of 0.1, and the experiment results are displayed in Fig. 8. It could be seen from Fig. 8 that as the value of rmp increases, the running times of our proposed method are almost the same, which proves that the change in rmp value has little effect on the running times. In addition, the proposed method obtains the superior experiment result in terms of deployment cost when rmp = 0.5. The reason can be summarized as follow. The same deployment cost based objective optimization function is used for all VMP optimization tasks. It indicates that the knowledge exchange between tasks is positive at a high probability during the process of evolution. And the larger value of rmp increases the probability of knowledge exchange between tasks, then obtains a better result. However, as the value of rmp continues to grow, the deployment cost may be gradually increased. The reason is that the larger value of rmp may cause the subpopulation specify to the tasks to fall into local optimum easily due to the loss of population diversity. In summary, the rmp is set to rmp = 0.5.
To test the effect of the number of , making = 100, 200, 500, 800, 1000, 1300, 1500, respectively, and the results are shown in Fig. 9. It can be seen from Fig 9 that, as the value of increases, the running time is increased and the deployment cost has also risen. This is due to the fact that the increase of expand the search space of the subpopulation from different tasks, resulting in the quality of the solution decreased. It is worth noting that the is not the smaller, the better. Although the smaller can bring advantages in the representation of individuals, the search space of the subpopulation specify to the tasks becomes narrow, and the subpopulation is easy to fall into local optimum. Summarily the was set to = 200.
VF Convergence Analysis
Take DS1 as an example, the convergence analysis of our proposed method is presented as follows. The situations on the other test data sets are similar. The maximum number of iterations is set as 100 in this experiment. Fig. 7 depicts the convergence curves of the proposed method and other compared algorithms. It can be seen from Fig. 7 that our method obtains better results than other compared algorithms in the first generation, which is crucial to guide the search process. And the convergence of our method is slowing down at the tenth iteration and approaching the local optima with deployment cost 2388.86. Actually, at the early iteration, the proposed method has converged to a superior local optima solution than other compared algorithms. The reason can be summarized as follows. On the one hand, the greedybased allocation operator is competent to finish the placement of VMs onto PSs, which is able to find superior local optimal solutions of the VMP optimization task at the early iteration. On the other hand, owing to the more effective representation block of the individual in the proposed method, the search space of each VMP optimization task is narrower and it is conducive to finding a local optima solution. The above results show that the proposed method can converge to a better local optima solution more fastly.
VG SingleFactorial Solver vs MultiFactorial Solver
In order to explore the effectiveness of multifactorial optimization technology, the multifactorial method proposed in this paper is modified to the singlefactorial version (SFEA) to compare. In this experiment, SFEA adopts the same initialization operator, crossover and mutation operator, greedybased allocation operator and environment selection operator as the proposed method. The difference between itself and the proposed multifactorial method is that SFEA lacks the process of establishing a unified express space. Meanwhile, SFEA is a singlefactorial solver and it has no intertask communication. For fair comparison, the same parameters as the proposed multifactorial method are used in SFEA. The test data sets DS1 to DS10 are conducted to assess the performance of SFEA, which the experiment result are summarized in Table V.
As seen in Table V, the running times of SFEA on all test instances are longer than its multifactorial version. This is attributed to the fact that the representation block of individuals in the multifactorial environment can not only decode into its related task, but also retain the genetic information of other tasks, which resulting in its effective. In the other three performance indicator (comprehensive utilization rate, the number of activated PSs and deployment cost), the proposed multifactorial method obtains superior results than SFEA on all the test data sets. The reason can be summarized as follows. The search space of population in SFEA is large and complex, which causes slower convergence speed of SFEA. Reversely, the search space of population in the multifactorial version is relatively narrow, so it can converge quickly. Besides these, the population of the multifactorial version is not easy to fall into local optimum due to the knowledge transfer appeared among tasks, which is also the key to obtain superior solutions than SFEA.
Data Set NO.  Size  Our Method  SFEA  
Time  Util  Num  Cost  Time  Util  Num  Cost  
DS1  5000  0.74s  85.47%  534.26  2316.15 





DS2  10000  1.48s  85.19%  1056.01  4578.87 





DS3  15000  2.27s  84.93%  1595.60  6918.30 





DS4  20000  3.02s  85.27%  2114.13  9174.47 





DS5  25000  3.81s  84.99%  2661.26  11540.25 





DS6  30000  4.65s  85.31%  3211.40  13926.30 





DS7  35000  5.42s  85.11%  3693.43  16026.43 





DS8  40000  6.25s  85.63%  4281.73  18578.08 





DS 9  45000  6.94s  85.42%  4759.33  20803.48 





DS10  50000  7.66s  85.01%  5215.50  22620.94 




VH Extra Largescale Test Data Sets
In this experiment, to further assess the performance of our proposed method on extra largescale VMP problems, the extra test data sets with a maximum size of 250,000 VMs is created as the same approach of the subsection V.A. Since OEMACS and FFD have been proved to be unsuitable for solving LVMP problems in the above experiments. In this experiment, the performance of our method is assessed in comparison with HGAPSO and RGGA. The experiment results are summarized in Table VI. As can be seen from Table VI, on the EDS6, the proposed method completes the assignment process for only 37.61s. The experiment results, in terms of the average comprehensive utilization rate, deployment cost and the average number of activated PSs, the proposed method also shows superior performance than HGAPSO and RGGA on all the extra test data sets.
Fig. 10 shows the average running time of the proposed method and compared algorithm on the extra largescale test data sets. It can be seen from Fig. 10, as the test data sets increases at a large step, the proposed method has the minimum and stability step of increased time, with the running time of only about 7s per 50,000 VMs added. The steps of HGAPSO and RGGA about increased time are relatively large, and their increased time step has a gradually increased tendency. The tuning time of HGAPSO is even longer on all the extra test data sets than RGGA. The experimental results proves that the proposed method can be really applied to deal with the problem of extra largescale VMP problems, and has the good expansibility and realtime performance.
Extra Data Set NO.  Size  Our Method  HGAPSO  RGGA  
Time  Util  Num  Cost  Time  Util  Num  Cost  Time  Util  Num  Cost  
EDS1  80000  12.38s  84.92%  8516.60  36915.98 









EDS2  100000  15.81s  85.06%  11615.23  46057.90 









EDS3  130000  19.64s  84.42%  13780.86  59762.06 









EDS4  150000  22.75s  85.76%  16054.73  69619.23 









EDS5  200000  30.28s  84.88%  21062.23  91355.30 









EDS6  250000  37.61s  85.65%  26453.77  114808.82 








Vi Conclusion and Future Work
This paper provides a MFO method to complete the placement of VMs onto PSs in heterogeneous environments of the largescale data center. We firstly reformulate a deployment cost based VMP problem in the form of the MFO problem. Multiple VMP optimization tasks based on deployment cost are modeled to achieve an organic composition of different configuration of PSs in heterogeneous environments, which lead to reduces the deployment costs directly for cloud providers. Furthermore, a multifactorial evolutionary algorithm coupled with the greedybased allocation operator is proposed to address the established MFO problem. After that, a remigration and merge operator is designed to obtain the integrated solution of the LVMP problem. The comprehensive experiments, including the largescale and extra largescale test data sets, are conducted to assess the effectiveness of our proposed method. The experimental result describes that the proposed method can significantly lower the optimization time and provide an effective placement solution for VMP problem in the largescale data center.
Although our proposed method shows promising problemsolving for the LVMP problems in heterogeneous environments, it only considered the optimization target of resource allocation. The actual data center is faced not only with the challenge in resource scheduling optimization but also included other complex optimization objectives, such as energy consumption optimization, network traffic optimization and so on. For future work, how to effectively integrate many different optimization objectives will continue to be explored based on MFO technology.
References
 [1] (200701) Improving packing algorithms for server consolidation.. In 2007 33th International Conference Computer Measurement Group, pp. 399–406. Cited by: §VB.
 [2] (2019) Incorporating ceteris paribus preferences in multiobjective virtual machine placement. IEEE Access 7 (), pp. 59984–59998. External Links: ISSN 21693536 Cited by: §I, §IIA.
 [3] (201507) A particle swarm optimization algorithm for poweraware virtual machine allocation. In 2015 6th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Vol. , pp. 1–6. External Links: Document, ISSN Cited by: §IIA.
 [4] (199704) Evolutionary computation: comments on the history and current state. IEEE Transactions on Evolutionary Computation 1 (1), pp. 3–17. External Links: Document, ISSN 1089778X Cited by: §I.

[5]
(2019)
Multifactorial evolutionary algorithm with online transfer parameter estimation: mfeaii
. IEEE Transactions on Evolutionary Computation (), pp. 1–1. External Links: ISSN 1089778X Cited by: §I.  [6] (200411) Power and energy management for server systems. Computer 37 (11), pp. 68–74. External Links: ISSN 00189162 Cited by: §I.
 [7] (19970701) Multitask learning. Machine Learning 28 (1), pp. 41–75. External Links: ISSN 15730565 Cited by: §I.
 [8] (2011) Tradeoffs between profit and customer satisfaction for service provisioning in the cloud. In Proceedings of the 20th International Symposium on High Performance Distributed Computing (HPDC), pp. 229–238. External Links: ISBN 9781450305525 Cited by: §I.
 [9] (20180401) Improving resource utilization via virtual machine placement in data center networks. Mobile Networks and Applications 23 (2), pp. 227–238. External Links: ISSN 15728153 Cited by: §I.
 [10] (201706) Performancemonitoringbased trafficaware virtual machine deployment on numa systems. IEEE Systems Journal 11 (2), pp. 973–982. External Links: Document, ISSN Cited by: §I.
 [11] (201604) Energyefficient virtual machines scheduling in multitenant data centers. IEEE Transactions on Cloud Computing 4 (2), pp. 210–221. External Links: ISSN Cited by: §I.
 [12] (199409) Simulated binary crossover for continuous search space. Complex Systems 9 (3), pp. 115–148. Cited by: §IIB.
 [13] (2011Sep.) Energyaware ant colony based workload placement in clouds. In 2011 12th International Conference on Grid Computing (GridCom), Vol. , pp. 26–33. External Links: Document, ISSN Cited by: §IIA.
 [14] (201710) Autoencoding evolutionary search with learning across heterogeneous problems. IEEE Transactions on Evolutionary Computation 21 (5), pp. 760–772. External Links: ISSN 1089778X Cited by: §I.
 [15] (201510) Memetic search with interdomain learning: a realization between cvrp and carp. IEEE Transactions on Evolutionary Computation 19 (5), pp. 644–658. External Links: ISSN 1089778X Cited by: §I.
 [16] (201901) Model predictive control for energyefficient, qualityaware, and secure virtual machine placement. IEEE Transactions on Automation Science and Engineering 16 (1), pp. 420–432. External Links: ISSN 15455955 Cited by: §I.
 [17] (2019) Evolutionary multitasking with dynamic resource allocating strategy. IEEE Transactions on Evolutionary Computation (), pp. 1–1. External Links: ISSN 1089778X Cited by: §I.
 [18] (201707) Multiobjective multifactorial optimization in evolutionary multitasking. IEEE Transactions on Cybernetics 47 (7), pp. 1652–1665. External Links: ISSN Cited by: §I.
 [19] (201606) Multifactorial evolution: toward evolutionary multitasking. IEEE Transactions on Evolutionary Computation 20 (3), pp. 343–357. External Links: ISSN 1089778X Cited by: §I, §IIB, §IIB.
 [20] (199511) Gaussian mutation and selfadaption for numeric genetic algorithms. In Proceedings of 1995 IEEE International Conference on Evolutionary Computation, Vol. 1, pp. 384. External Links: ISSN Cited by: §IIB.
 [21] (201502) Efficient multitenant virtual machine allocation in cloud data centers. Tsinghua Science and Technology 20 (1), pp. 81–89. External Links: Document, ISSN Cited by: §I.
 [22] (200602) ParEGO: a hybrid algorithm with online landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation 10 (1), pp. 50–66. External Links: ISSN 1089778X Cited by: §I.
 [23] (201910) Evolutionary multitasking sparse reconstruction: framework and case study. IEEE Transactions on Evolutionary Computation 23 (5), pp. 733–747. External Links: ISSN Cited by: §I.
 [24] (201311) Migrationbased virtual machine placement in cloud systems. In 2013 IEEE 2nd International Conference on Cloud Networking (CloudNet), Vol. , pp. 83–90. External Links: ISSN Cited by: §I.
 [25] (201806) Holistic virtual machine scheduling in cloud datacenters towards minimizing total energy. IEEE Transactions on Parallel and Distributed Systems 29 (6), pp. 1317–1331. External Links: ISSN Cited by: §I.
 [26] (201512) Virtual machine placement strategy based on discrete firefly algorithm in cloud environments. In 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Vol. , pp. 61–66. External Links: Document, ISSN Cited by: §IIA.
 [27] (201006) Generalizing surrogateassisted evolutionary computation. IEEE Transactions on Evolutionary Computation 14 (3), pp. 329–355. External Links: ISSN 1089778X Cited by: §I.
 [28] (201802) An energy efficient ant colony system for virtual machine placement in cloud computing. IEEE Transactions on Evolutionary Computation 22 (1), pp. 113–128. External Links: ISSN 1089778X Cited by: §I, §IIA, §VB.
 [29] (201407) Energy aware virtual machine placement scheduling in cloud computing based on ant colony optimization approach. pp. . External Links: ISBN 9781450326629, Document Cited by: §IIA.
 [30] (2018) Reliable virtual machine placement based on multiobjective optimization with trafficaware algorithm in industrial cloud. IEEE Access 6 (), pp. 23043–23052. Cited by: §I.
 [31] (201307) Probabilistic consolidation of virtual machines in selforganizing cloud data centers. IEEE Transactions on Cloud Computing 1 (2), pp. 215–228. External Links: ISSN 21687161 Cited by: §I.
 [32] (201710) A study of the performance of a cloud datacenter server. IEEE Transactions on Cloud Computing 5 (4), pp. 590–603. External Links: ISSN 21687161 Cited by: §I.
 [33] (2016) Evolutionary multitasking: a computer science view of cognitive multitasking. Cognitive Computation 8 (2), pp. 125–142. Cited by: §IIB.

[34]
(201010)
A survey on transfer learning
. IEEE Transactions on Knowledge and Data Engineering 22 (10), pp. 1345–1359. External Links: ISSN 10414347 Cited by: §I.  [35] (2018) A multiobjective ant colony system algorithm for virtual machine placement in traffic intense data centers. IEEE Access 6 (), pp. 58912–58923. External Links: Document, ISSN Cited by: §I.
 [36] (201506) Energy efficient virtual machine placement algorithm with balanced resource utilization based on priority of resources. Computer Engineering and Applications Journal 4, pp. 107–118. Cited by: §I.
 [37] (200701) A genetic algorithm with exon shuffling crossover for hard bin packing problems. In 2007 Genetic and Evolutionary Computation Conference (GECCO), pp. 1365–1371. Cited by: §IVD.
 [38] (201901) Multiobjective energy efficient virtual machines allocation at the cloud data center. IEEE Transactions on Services Computing 12 (1), pp. 158–171. External Links: Document, ISSN 19391374 Cited by: §IIA, §VB.
 [39] (201901) Energyefficient adaptive resource management for realtime vehicular cloud services. IEEE Transactions on Cloud Computing 7 (1), pp. 196–209. External Links: ISSN 21687161 Cited by: §I.
 [40] (201301) A twotiered ondemand resource allocation mechanism for vmbased data centers. IEEE Transactions on Services Computing 6 (1), pp. 116–129. External Links: Document, ISSN Cited by: §I.
 [41] (201807) Effective multifactorial evolutionary algorithm for solving the cluster shortest path tree problem. In 2018 IEEE Congress on Evolutionary Computation (CEC), Vol. , pp. 1–8. External Links: ISSN Cited by: §IIB.
 [42] (201706) Energy efficient vm placement for effective resource utilization using modified binary pso. Computer Journal 61 (6), pp. 832–846. Cited by: §IIA.
 [43] (200712) Computing in the clouds. netWorker 11 (4), pp. 16–25. External Links: ISSN 10913556 Cited by: §I.
 [44] (201106) Solving virtual machine packing with a reordering grouping genetic algorithm. In 2011 IEEE Congress of Evolutionary Computation (CEC), Vol. , pp. 362–369. External Links: Document, ISSN 19410026 Cited by: §IIA, §IVD, §VB.
 [45] (201208) Green wireless communications: from concept to reality [industry perspectives]. IEEE Wireless Communications 19 (4), pp. 4–5. External Links: ISSN Cited by: §I.
 [46] (201406) Energy efficient multiresource allocation of virtual machine based on pso in cloud data center. Mathematical Problems in Engineering 2014 (), pp. 1–8. External Links: ISSN Cited by: §I, §IIA.
 [47] (201604) EnReal: an energyaware resource allocation method for scientific workflow executions in cloud environment. IEEE Transactions on Cloud Computing 4 (2), pp. 166–179. External Links: ISSN 21687161 Cited by: §I, §IIA.
 [48] (201907) Multitasking multiobjective evolutionary operational indices optimization of beneficiation processes. IEEE Transactions on Automation Science and Engineering 16 (3), pp. 1046–1057. External Links: ISSN 15455955 Cited by: §IIB.
 [49] (201712) Twostage assortative mating for multiobjective multifactorial evolutionary optimization. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), Vol. , pp. 76–81. External Links: Document, ISSN Cited by: §IIB.
 [50] (2017) Energyefficient manyobjective virtual machine placement optimization in a cloud computing environment. IEEE Access 5 (), pp. 16006–16020. External Links: Document, ISSN Cited by: §I.
 [51] (2018Secondquarter) A survey on virtual machine migration: challenges, techniques, and open issues. IEEE Communications Surveys Tutorials 20 (2), pp. 1206–1243. External Links: ISSN 1553877X Cited by: §I.
 [52] (20100501) Cloud computing: stateoftheart and research challenges. Journal of Internet Services and Applications 1 (1), pp. 7–18. External Links: ISSN 18690238 Cited by: §I.
 [53] (2019) Selfregulated evolutionary multitask optimization. IEEE Transactions on Evolutionary Computation (), pp. 1–1. External Links: Document, ISSN 1089778X Cited by: §I.
 [54] (201612) Evolutionary multitasking in combinatorial search spaces: a case study in capacitated vehicle routing problem. In 2016 IEEE Symposium Series on Computational Intelligence, Vol. , pp. 1–8. External Links: ISSN Cited by: §IIB.
Comments
There are no comments yet.