With the development of new agile technologies such as 5G (and beyond), the number of connected devices are increasing at a rapid pace. However, the lack of direct application program interface (API) interactions with network services impose limitations on the handling of the rapidly changing traffic patterns. Also, the rise of cloud, Internet of Things (IoT) and AI-driven applications have made this task more challenging. To cope with the ever-increasing demands of networks in a cost-efficient manner, the conventional networks are demanding more flexible and dynamic solutions [1, 2]. Towards this end, network softwarization is gaining huge popularity which supports rapid provisioning and deployment of network services to meet QoS requirements . By reducing overhead, automating infrastructure and increasing maintainability, it aims to build agile and flexible networks to provision convergent networking solutions.
In this direction, Software Defined Networks (SDN) and Network Functions Virtualization (NFV) have been introduced . SDN is an exciting and evolving paradigm that turns the network into a flexible and programmable platform able to optimally utilize the resources. It separates the control plane of a network from the data plane to make it centrally manageable . NFV, on the other hand, represents the network service as a series of virtual network functions (VNFs). In NFV architectures, individual VNFs can be chained together to perform the desired sequence, known as service function chain (SFC). Using SFC, a large number of VNFs can be connected together in an NFV environment. To support a fully virtualized network, VNFs provide network scalability and agility, while enabling better provisioning of network resources .
To meet the dynamic requirements of the network resources, several solutions have been proposed in the literature for SFCs deployment. For example, Chen et al.  proposed a fully decentralized online approach for service chaining based on stochastic dual-gradient method. Similarly, Sun et al.  studied the problem of SFC orchestration using a mesh aggregation approach where a feedback mechanism was deployed to improve its success rate. In another work, Sun et al. 
In the similar context, Medhat et al.  explored the limitations of current SFC approaches in next-generation networks and showed that approaches integrating SDN and NFV technologies have higher SFC scalability and flexibility compared to others. To ensure the benefits of the service-chaining policy, Ding et al.  proposed OpenSCaaS platform by integrating SDN and NFV. Furthermore, Zhang et al.  presented a framework to construct efficient SFCs by combining SDN and NFV. Bruschi et al.  proposed a softwarized SDN/NFV approach to support the cloud services in a more scalable and sustainable way. Likewise, Pie et al.  developed a novel routing algorithm to solve the SFC problem in SDN and NFV-enabled network. In order to minimize the resource consumption flow costs with SFC requests, binary integer programming was used. Although several attempts have been made to tackle the service deployment of networks, they all rely on single domain SDN networks. However, the rapidly growing network traffic and constant enlargement of network scales pose challenges to single domain SDN architectures in terms of latency, overhead, fault tolerance and load balancing, thereby affecting the overall QoS experience for end-users.
To overcome these challenges, large-scale SDN networks have been proposed. However, the deployment of network services on a large-scale SDN network involves the distribution of VNFs on multiple SDN domains. Thus, the service provisioning across these networks requires the collaboration among different SDN controllers which results in additional network overhead. However, the deployment of network services across multi-domain SDNs is still in its early phases . Further, the deployment of network services in multi-domain SDN networks requires an efficient allocation of resources for different service requests at a reduced cost. Thus, it is of primary interest to implement energy efficient solutions which dynamically adapt the usage of resources according to the demands.
This paper proposes an energy-efficient approach to deploy NFVs in multi-domain SDN networks. To the best of our knowledge, the evaluation of energy characteristics in multi-domain SDN setups is a novel research initiative and has not been explored. The key contributions of the proposed wok are summarized as follows:
We analyze the problem of VNF deployment in multi-domain SDN environments (associated with energy consumption and resource allocation challenges), and mathematically formulate a multi-objective optimization problem for the effective deployment of NFV in SDN. The formulated problem aims at achieving an optimal mapping of VNFs across a multi-domain SDN setup while minimizing the overall energy consumption level.
We then evaluate the proposed multi-objective problem on different evolutionary algorithms to have an optimal solution in polynomial time using the widely used MOEA framework. To attain this objective, different variants of Non-dominated Sorting Genetic Algorithm (NSGA) are used namely NSGA-II, NSGA-III and NSGA-II.
We then extensively assess the performance of the proposed scheme using the most appropriate evolutionary algorithm under different scenarios in terms of number of VNFs, number of SDN domains, number of constraints and decision variables.
The rest of this paper is organized as follows. Section II presents the system model for the proposed scheme. In Section III, the problem of VNF deployment in multi-domain SDN is formulated. Finally, in Section IV, the formulated problem is subjected to different variants of NSGA in different networks scenarios. Finally, this work is concluded in Section V with future directions.
Ii System Model
In the following section, we detail the system model for the next-generation communication systems empowered by SDN and NFV. The designed solution helps in the optimized mapping of the VNFs to the available SDN nodes (scattered across multiple domains) keeping the energy consumption and resource utilization profile in check.
Fig. 1 depicts the system model of the proposed communication system. It represents the coupling of SDN with NFV for seamless networking operations. SDN is a software-based programming paradigm for modern networks that rely on software reconfigurations for changing the network policies. As shown in the figure, the SDN platform is comprised of a data plane and a control plane. Here, the data plane consists of the Open-Flow (OF)-enabled routers and switches with dedicated flow table entries. These OF-flow tables are primarily controlled by the SDN’s control plane via the southbound API, i.e., OpenFlow protocols. In other words, the policy changes are assumed by the SDN’s controller; which are reproduced at the data plane .
On the other hand, NFV is an upcoming networking paradigm that helps to segregate the network functions from the underlying proprietary hardware. Further, it helps to execute the network functions on software with flexible implementation which are referred to as VNFs. By coupling NFV with the powerful paradigm of SDN, the network services can be established with ease by redirecting the traffic through the appropriate VNFs in a sequential manner. The Internet Engineering Task Force (IETF) defines the abstract notion of VNFs using SFC–a means to implement VNFs. These SFCs are managed by the SDN’s centralized controller and deployed on the underlying hardware. With the SDN-NFV coupling, the deployment of SFCs becomes flexible without the need to change the hardware.
However, the constant enlargement of networks has resulted in wide-scale SDNs with several interconnected domains referred to as multi-domain SDNs . All the domains are individually administrated by their respective controllers and are linked with each other to provide the required network services. The provision of these network services involves the successful deployment of VNFs (belonging to different SCFs) across different SDN domain nodes (physical or virtual). Nevertheless, the collaboration between different SDN domain controllers is essential for the efficient distribution of resources amongst the VNFs. The same can be achieved using east and west-bound APIs. In summary, the focus of the proposed work is to manage the underlying resources effectively while attaining minimal energy consumption. In the proposed work, the same is achieved by mapping VNFs to the available array of SDN domains while assigning the required resources (CPU cores, memory and storage) to VNFs for their execution. The details of the overall deployment process are illustrated using Fig. 2.
Iii Problem Formulation
In this section, the problem of VNF deployment across a multi-domain SDN set-up is illustrated. The key objective functions of the considered problem followed by their corresponding constraints are presented.
A. List of Objective Functions
As discussed earlier, the proposed scheme supports two major objective functions () as illustrated under. Here, denotes the decision variable expressed as follows:
In the above equation, the nature of is binary, i.e., . Here, indicates that the VNF is deployed successfully on the SDN domain, while suggests that VNF is not deployed on the SDN domain.
1) Maximum deployment of VNFs: One of the foremost objectives of the proposed scheme is to ensure maximum deployment of VNFs on the available array of SDN nodes. This is attained by using a maximization operation over the summation of as follows:
In the above equation, denotes the total number of VNFs to be deployed.
2) Minimum Energy Consumption: This objective function deals with the minimization of the overall energy consumption involved in executing VNFs on the available SDN domains. It is denoted as follows:
where, denotes the total number of SDN domains and variable refers to the energy consumption (in kWh) associated with the execution of the VNF over the SDN domain.
B. List of Constraints
This section presents different set of constraints that should be respected while achieving energy-aware deployment of VNFs in the multi-domain SDN setup.
1) VNF deployment constraint: The following constraint ensures that only a single VNF is allocated at most a single SDN domain as expressed using the below mentioned equation. The said constraint restricts the multiple deployment of an individual VNF in multi-domain SDN environments and helps to minimize the overall energy consumption.
2) Resource restriction with respect to CPU cores: The following constraint ensures that VNF deployment on different physical nodes always meets their required CPU core needs. The said restriction helps to allocate the requested number of cores to the mapped VNFs using the following equation. In other words, the constraints ensure that no more VNFs can be mapped to a physical node without the required number of CPU cores requested by the corresponding VNF.
In the above equation, refers to the number of the CPU cores required by the VNF and denotes the upper limit on the node’s CPU core capacity.
3) Resource restriction with respect to Memory: In line, with the above mentioned constraint, the following restriction imposes the upper limit of the main memory that could be allocated to the VNFs on a particular SDN domain node.
here, variables and represent the main memory required by the VNF and the upper limit on the main memory capacity of the node, respectively.
4) Resource restriction with respect to Storage: The following equation represents the resource restriction corresponding to disk storage allocation.
The variable defined in the above equation denotes the disk storage requested by the VNF; while the variable denotes the upper limit on disk storage for the node.
C. Overall Problem at hand
The overall problem can be summarized as follows:
As detailed above, the considered problem comprises of the two objective functions and . The minus sign before denotes maximization operation, i.e., the aim is to maximize the deployment of VNFs. On the other hand, depicts the minimization operation over the energy consumption objective function. The associated constraints span from Eq. (4) till Eq. (7). It is worth noticing here that the nature of the problem mentioned above is linear with linear constraints; which suggests that the problem under consideration is ILP with binary decision variable . However, solving an ILP with a huge number of parameters (say thousands of ) is quite a gigantic task. This is because with increasing Internet traffic, multi-domains SDNs are expected to grow exponentially, which in turn would lead to a substantial increase in the number of VNFs () and SDN-domains (). Consider the following example,
|No. of Constraints|
Thus, with the constant increase in parameters, an optimal solution for the above problem would be time-consuming and impossible in polynomial time. Henceforth, the proposed scheme leverages the advantages of a popular multi-objective evolutionary algorithm called -NSGA-II. It is characterized by its fast convergence, low overall complexity, explicit diversity preservation capability, epsilon-dominance, adaptive population sizing, and automatic termination [18, 19].
The experimental evaluation against the existing state-of-the-art multi-objective evolutionary algorithms demonstrate that -NSGA-II is best suited for our problem, as justified by the preliminary results demonstrated in the upcoming section.
Iv Results and Discussions
This section illustrates the experimental evaluation of the proposed scheme compared to different state-of-the-art schemes on different evaluation parameters.
To evaluate the proposed VNF deployment, the widely accepted MOEA Framework has been employed . It is an open source Java library which is heavily used by the research community for designing and evaluating different multi-objective evolutionary algorithms such as
-NSGA-II, NSGA-II, NSGA-III, generalized differential evolution (GDE3), strength pareto evolutionary algorithm 2 (SPEA2), indicator-based evolutionary algorithm (IBEA), speed-constrained multi-objective particle swarm optimization (SMPSO) covariance matrix adaptation evolution strategy (CMA-ES), multi-objective evolutionary algorithm with decomposition (MOEA/D), and-MOEA. Additionally, it also supports rapid design and development of customized problems and algorithms. For the proposed scheme, the formulated multi-objective optimization problem (defined Eq. (III)) has been scripted in java using MOEA Framework and has been evaluated with existing state-of-the-art algorithms such as NSGA-II, NSGA-III, and -NSGA-II.
The experiment evaluation for the proposed multi-objective scheme was carried out across two phases (referred to as Case Studies). In the first phase of the assessment, the proposed scheme was subjected to different multi-objective evolutionary algorithms, and their relative performance was compared considering Pareto front and elapsed time. Based on the comparison, it was concluded that -NSGA-II was the most appropriate algorithm to address the proposed NFV deployment across multi-domain SDN setups. Accordingly, in the next phases, the efficacy of the proposed scheme was evaluated using -NSGA-II on a dataset spanning different scenarios. The detailed description of these phases is illustrated below.
A. Case Study I
The problem of VNF deployment in multi-domain environments has been evaluated using different versions of NSGA namely NSGA-II, NSGA-III, and -NSGA-II. The related results are summarized in Fig. 3. Each of these algorithms has been evaluated with 10,000 function evaluations with 20 seeds in the MOEA framework’s Diagnostic Tool. The related results regarding pareto front and elapsed time are summarized in Fig. 2(a) and Fig. 2(b), respectively. The pareto front evaluations show that -NSGA-II achieves the most optimal performance followed by NSGA-II and NSGA-III; in terms of trade-off between the two objective function and . Although the performance of -NSGA-II and NSGA-II is almost identical; however the former results in much less energy consumption deployments as compared to the other. Additionally, the computational time of the three algorithms is summarized in Fig. 2(b). Similar to the previous case, -NSGA-II achieves the optimal performance with a reduced value of elapsed time. This suggests that -NSGA-II achieves the optimal performance with quick computational processing.
Thus, for both parameters, -NSGA-II stands out which makes it the ideal choice to solve the considered VNF deployment challenge in a multi-domain SDN setup.
B. Case Study II
During this case study, the performance of the proposed multi-objective approach for VNF deployment has been evaluated using -NSGA-II under different scenarios in terms of number of VNFs deployed, number of SDN domain nodes, number of decision variables and number of constraints. The variability concerning the first two parameters can be understood from the data plotted in Figs. 3(a) and Fig 3(b). For the other two parameters, the following equations can be used for the computational purpose.
|No. of Decision Variables|
|No. of Constraints|
In order to achieve successful deployment of VNFs in a multi-domain setup, Eq. (8) was evaluated using -NSGA-II. The related results in terms of successful VNF deployment and energy consumptions statistics are highlighted in Fig. 3(c) and Fig. 3(d), respectively. The obtained results evidently support the fact that the proposed scheme achieves an optimal trade-off between the two competing objective functions and . Overall, in comparison with the existing NSGA variants, the proposed approach using -NSGA-II gives better performance regarding energy consumption and successful VNF deployment.
Gradual evolutions in the networking infrastructures have resulted in the convergence of SDN and NFV to provide seamless deployment of network services. Additionally, this convergence has led to the amplification of SDNs from single domain to multiple domains. Under such multi-domain SDNs, the realization of NFV would become a challenging task particularly due to the complex interactions between SDN controllers of different domains. Particularly, the energy consumption management will also evolve as a challenging task. Thus, in this paper, the problem of NFV deployment in multi-domain SDNs has been addressed. The main focus of the proposed work was to minimize the energy consumption caused by VNF deployment across multiple domain SDN while achieving their optimal deployment with the required number of resources. In summary, the said problem has been modeled as an ILP and solved using different variants of NSGA-a popular evolutionary optimization algorithm. However, experimental evaluations suggests that -NSGA-II gives the optimal trade-off between the considered objective functions and is best suited for the problem under consideration. Additionally, the experimental results also suggest that the proposed VNF deployment solution also reduces the energy consumption to a large extent.
In the near future, we will extend this work to locality-based VNF deployment; where VNFs of a particular SFC would be deployed in a specific SDN domain.
-  A. Singh, S. Garg, S. Batra, N. Kumar, and J. J. Rodrigues, “Bloom filter based optimization scheme for massive data handling in IoT environment,” Future Generation Computer Systems, vol. 82, pp. 440–449, 2018.
-  S. Garg, A. Singh, K. Kaur, S. Batra, N. Kumar, and M. S. Obaidat, “Edge-Based Content Delivery for Providing QoE in Wireless Networks Using Quotient Filter,” in IEEE International Conference on Communications (ICC), Kansas City, USA, May 2018.
-  K. Kaur, S. Garg, G. S. Aujla, N. Kumar, J. J. Rodrigues, and M. Guizani, “Edge Computing in the Industrial Internet of Things Environment: Software-Defined-Networks-Based Edge-Cloud Interplay,” IEEE Communications Magazine, vol. 56, no. 2, pp. 44–51, 2018.
-  V. Nguyen, A. Brunstrom, K. Grinnemo, and J. Taheri, “SDN/NFV-Based Mobile Packet Core Network Architectures: A Survey,” IEEE Communications Surveys Tutorials, vol. 19, no. 3, pp. 1567–1602, 2017.
-  R. Mijumbi, J. Serrat, J. Gorricho, N. Bouten, F. D. Turck, and R. Boutaba, “Network Function Virtualization: State-of-the-Art and Research Challenges,” IEEE Communications Surveys Tutorials, vol. 18, no. 1, pp. 236–262, 2016.
-  X. Chen, W. Ni, T. Chen, I. Collings, X. Wang, R. P. Liu, and G. B. Giannakis, “Multi-Timescale Online Optimization of Network Function Virtualization for Service Chaining,” IEEE Transactions on Mobile Computing, 2018, DOI: 10.1109/TMC.2018.2885301.
-  G. Sun, Y. Li, D. Liao, and V. Chang, “Service Function Chain Orchestration Across Multiple Domains: A Full Mesh Aggregation Approach,” IEEE Transactions on Network and Service Management, vol. 15, no. 3, pp. 1175–1191, 2018.
-  G. Sun, G. Zhu, D. Liao, H. Yu, X. Du, and M. Guizani, “Cost-Efficient Service Function Chain Orchestration for Low-Latency Applications in NFV Networks,” IEEE Systems Journal, pp. 1–13, 2018, DOI: 10.1109/JSYST.2018.2879883.
-  A. Gupta, M. F. Habib, U. Mandal, P. Chowdhury, M. Tornatore, and B. Mukherjee, “On Service-Chaining Strategies using Virtual Network Functions in Operator Networks,” Computer Networks, vol. 133, pp. 1–16, 2018.
-  A. M. Medhat, T. Taleb, A. Elmangoush, G. A. Carella, S. Covaci, and T. Magedanz, “Service function chaining in next generation networks: State of the art and research challenges,” IEEE Communications Magazine, vol. 55, no. 2, pp. 216–223, 2017.
-  W. Ding, W. Qi, J. Wang, and B. Chen, “OpenSCaaS: an open service chain as a service platform toward the integration of SDN and NFV,” IEEE Network, vol. 29, no. 3, pp. 30–35, 2015.
-  J. Zhang, Z. Wang, N. Ma, T. Huang, and Y. Liu, “Enabling Efficient Service Function Chaining by Integrating NFV and SDN: Architecture, Challenges and Opportunities,” IEEE Network, vol. 32, no. 6, pp. 152–159, 2018.
-  R. Bruschi, F. Davoli, P. Lago, A. Lombardo, C. Lombardo, C. Rametta, and G. Schembra, “An SDN/NFV Platform for Personal Cloud Services,” IEEE Transactions on Network and Service Management, vol. 14, no. 4, pp. 1143–1156, 2017.
-  J. Pei, P. Hong, K. Xue, and D. Li, “Resource Aware Routing for Service Function Chains in SDN and NFV-Enabled Network,” IEEE Transactions on Services Computing, 2018, DOI: 10.1109/TSC.2018.2849712.
-  C. Zhang, X. Wang, F. Li, M. Huang, and Q. He, “Network service chains deployment across multiple SDN domains,” International Journal of Communication Systems, vol. 31, no. 18, p. e3826, 2018.
-  R. Chaudhary, N. Kumar, and S. Zeadally, “Network service chaining in fog and cloud computing for the 5G environment: Data management and security challenges,” IEEE Communications Magazine, vol. 55, no. 11, pp. 114–122, 2017.
-  P. Reed, B. S. Minsker, and D. E. Goldberg, “Simplifying multiobjective optimization: An automated design methodology for the nondominated sorted genetic algorithm-II,” Water Resources Research, vol. 39, no. 7, 2003.
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist
multiobjective genetic algorithm: NSGA-II,”
IEEE transactions on evolutionary computation, vol. 6, no. 2, pp. 182–197, 2002.
-  D. Hadka, “MOEA Framework: A Free and Open Source Java Framework for Multiobjective Optimization,” [Accessed on: Nov. 2018]. [Online]. Available: http://moeaframework.org