One of the essential characteristics of Dynamic Multi-objective Optimization Problems (DMOPs)  is that the optimization functions will change with environments or times, and it is of great significance for many practical applications 
. Therefore, efficiently solving DMOPs has become an important direction in the field of evolutionary computation community. For example, when designing a missile, the defense department should consider its range, precision, weight and fuel consumption. This is a four-objective optimization problem, and these optimization objectives will vary depending on environments and times. However, Most of the existing methods do not perform well when dealing with such problems. The main difficulty is how to quickly track the changing Pareto optimal front (POF).
In recent years, great progress has been made in this field, and many different algorithms have been proposed. Among those algorithms, the methods based on prediction have attracted special attention, and the basic idea of the method is to solve DMOPs by reusing “experience” effectively. For example, Muruganantham et al. 
proposed a Kalman Filter prediction based DMOEA(MOEA/D-KF) that utilizes a prediction model based on Kalman Filter based on the multiobjective EA with Decomposition.
In this paper, we argue that Incremental Support Vector Machine (ISVM)  can be used to train a prediction model, and the model helps any kind of population-based DMOPs algorithms by generating a high-quality initial population. This method has unique advantages in dealing with DMOPs, because it directly establishes connections between different POS, without requiring more computational resources to repeatedly train SVM classifiers.
The contribution of this research is to propose an algorithm which combining evolutionary multi-objective optimization algorithm with the ISVM technique. This combination has the following two advantages. First, the proposed design can improve the search accuracy in the way of reusing past experience. Second, this method trains the SVM classifier in an online manner, and then obtains the initial population for the next time, which can make more efficient use of computing resources.
The rest of this paper is organized as follows: In Section II, we will introduce some basic concepts of dynamic optimization problems, Incremental Support Vector Machine ( ISVM ) and related work. In Section III, we will propose the Incremental Support Vector Machine based Dynamic Multi-Objective Evolutionary optimization Algorithm, ISVM-DMOEA. In Section IV we firstly introduce the evaluation criteria, test examples and comparative methods, and then experimental results are analyzed. In Section V, we conclude the main work of this research and the future research direction are discussed.
Ii Preliminaries and Related Works
Ii-a Concepts of Dynamic Multi-objective Optimization
For dynamic multi-objective optimization problems, its optimization functions often vary with times or environments. Mathematically, a DMOP can be described as:
where represents time or environment index and is the decision vector. is the number of objectives, which is the objective space.
[Dynamic Decision Vector Domination] At a particular moment , a decision vector is Pareto dominated by another vector , it can be expressed as , if and only if:
[Dynamic Pareto-optimal Set] At time t, if and only if there is no other decision vector can Pareto dominate decision vector , the is called Pareto optimal solution. All Pareto optimal solutions make up Dynamic Pareto-optimal Set(DPOS) at time , that is:
[Dynamic Pareto-optimal Front] The Dynamic Pareto-Optimal Front (DPOF) is the set of corresponding objective vectors of the Pareto-optimal Solution at time .
Ii-B Incremental Support Vector Machines
Support Vector Machine ( SVM ) was proposed in 1964 , developed rapidly after the 1990s and spawned a series of improved and extended algorithms. The SVM becomes a well-known learning method used for classification problems 
, and it is a generalized linear classifier for binary classification of data in supervised learning. Its decision boundary is the maximum-margin hyperplane that is solved for learning samples. Incremental Support Vector Machine (ISVM) is the combination of Online learning technology and the SVM. Incremental technology has been developed to facilitate bulk SVM learning on very large data sets and has been widely used in the SVM community.
Given the training data and learning objective
in the classification problem, where each sample of training data contains multiple features and thus constitutes a feature space. The learning objective is a binary variablerepresenting negative class and positive class. A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane, which can be described as:
where and .
In linear inseparable problems, the use of SVM with hard margins will generate classification errors, so a new optimization problem can be constructed by introducing loss function on the basis of maximizing margins. The optimization problem of soft margin SVM is shown as follows:
Where is a constant and the second term of Equation (2) provides an upper bound for the error in the training data, and the first term makes maximum margin of separation between classes.
when learning nonlinear SVMs, to simplify matters and then this quadratic program is typically expressed in its dual form:
Where , is the given kernel function which to implicitly map into a higher (possibly infinite) dimensional feature space.
The Karush-Kuhn-Tucker (KKT) condition uniquely defines the solution of dual parameters , and minimizes the form (2):
The KKT conditions partition the training data into three parts:
, the set S of margin support vectors;
, the set E of error support vectors;
, the set R of the remaining vectors.
When we increment the unlearned examples into the solution, our goal will be to keep all previously seen training data in KKT conditions simultaneously.
Ii-C Related Works
The DMOPs field has made great progress, and existing algorithms can be generally divided into the following categories: Diversity based Approaches, parallel approaches, Memory based Approaches and Change prediction based Approaches .
An important work of diversity based approaches is the Dynamic NSGA-II (DNSGA-II) proposed by  in 2006. Deb et al. extended NSGA-II to deal with DMOPs by introducing diversity in each change detection. If the target or constraint violation value has changed, the problem is considered to have changed. Then, all outdated solutions (that is, reassessment). This process allows for the use of changing objectives and constraint functions to evaluate both offspring and parent solutions. In , Chen proposed to maintain genetic diversity by regard it as an additional objective when solving multi-objective optimization. They presented the Individual Diversity Evolutionary Method (IDEM) to add a useful selection pressure for optimizing PS and maintaining diversity . The results show that the algorithm can converge to the optimal solution effectively and track the change of PFs while maintaining the diversity of solution sets.
Parallel EAs utilize several subpopulations that evolve simultaneously under different processors and then communicate some informations in a structured network . In , Camara et al. proposed a method to apply parallel single pre-genetic algorithm (PSFGA) to dynamic environment. PSFGA is a master-slave architecture algorithm, which divides the population into subpopulations which performs a certain amount of generations and preserve only non-dominant solutions, and then the main process joins all the solutions to a new population.
The memory-based approach uses additional memory to implicitly or explicitly store useful information from past generations to guide future searches. It has been proved that when the optimal solution returns to the previous position repeatedly or the environment changes periodically, this algorithm will help save computing time and bias search process, thus becoming very efficient. In , Azzouz et al. proposed an adaptive hybrid population management strategy, which is based on a technology that can measure the severity of environmental changes, then according to the technology it can adjust memory, local search (LS) and the number of random solutions.
When the behavior of dynamic problems follows a certain trend, a prediction model can be used. In 2014, the author defined a new prediction model  to solve DMOPs with Translational optimal POS (DMOP-tps). Dmop-tps is a specific type of DMOP in which POS is converted periodically over time. When the environment changes, the strategy proposed by Deb et al.  is used to detect the changes. Then, the population is re-initialized according to the properties of DMOP. Muruganantham et al.  proposed a Kalman Filter prediction based DMOEA(MOEA/D-KF) based on the multiobjective EA with Decomposition. When the change of environment is detected, kalman filter is applied to the whole population, which leads the search to the new pareto optimal solution in the decision space. The influence of the severity and frequency of change on the performance of the this algorithms is studied.
Iii Incremental Support Vector Machine Based Dynamic Multi-Objective Optimization Algorithm
A dynamic multi-objective optimization problem is based on the fact that different environments follow different possible distributions, and these are not independent, but interrelated. For the dynamic multi-objective optimization problem, it is easy to get the solution. However, it is difficult for us to judge whether this solution is good or bad, but the solution already obtained should contain useful information, which can be used to predict POS at the next moment. So in this article, we turn the decision problem into a classification problem. In other words, we are constantly training the ISVM classifier through the information we have obtained, that is, the POS of past moments.
At the same time, a good initial population is the key to solving the dynamic multi-objective problem. A good initial population not only speeds up the solution, but also improves the quality and accuracy of the solution. By training the SVM classifier in an online way, we can use this continuously improved classifier to filter out the solutions, and then generate the initial population at the next moment.
In this article, we propose a incremental support vector machine (ISVM) based dynamic optimization algorithm. The details of the proposed algorithm is descried in Algorithm. 1.
This algorithm is mainly divided into two modules. At the initial moment, we combine the solution (positive example) in POS obtained by using multi-objective evolutionary algorithm and the randomly generate some solutions of the problem (non-POS, negative example) into training samples, and then obtain a SVM classifier . When the environment changes, we regard the changes as the incremental learning process of SVM classifier for the change process of dynamic multi-objective optimization problems. At each change moment, we consider the obtained POS and non-POS as samples to update the parameters of classifier. At the same time, using classifier, we can classify the solutions in the next moment into two categories, “good” and “bad”. And the good solutions is reserved as the initial population of the next moment. Finally, we can get the classification model with the best performance.
In this research, we only used the NSGA-II, the MOPSO and the RM-MEDA algorithms to obtain POS at next time, but in fact, any population-based algorithm can use our proposed method to achieve performance improvements.
The at particular moment are obtained by while are randomly generate,and they have the same number.
At the initial moment, we train a SVM classifier by using and , and when t change from 1 to n, we still train the same using and .
Iv Empirical Study
In general, our proposed algorithm is applicable to any kind of population-based optimization algorithm. In our experiment, we chose three representative experiments to prove our method. The first multiobjective optimization algorithm is based on particle swarm optimization , it was called MOPSO for short. The second one is the NSGA-II and it is a multiobjective genetic algorithm that applies nondominated sorting and crowding distance. The third one is a distribution estimation algorithm 
based on global statistical information to construct a probability model, We simply refer to it as the RM-MEDA. Obviously, these three algorithms belong to different categories, and they were not originally designed for dynamic optimization. But they are well developed, so we can improve our level of persuasion and confidence in the technologies we propose. The three corresponding algorithms with incremental SVM are called ISVM-MOPSO, ISVM-NSGA-II, and ISVM-RMMEDA respectively. It is worth noting that parameters, such as population size, iteration number are all the same. In other words, for the three groups of algorithms, we did not deliberately adjust the experimental parameters for getting better performances.
Iv-a Performance Metrics, Testing Functions and Settings
In this study, when comparing with other competitive algorithms, we use inversed generational distance (IGD) and its variants as performance indicators to evaluate the quality of solutions obtained by different algorithms.
The inverted generational distance (IGD)  is an index to measure the distance between the real Pareto Optimal Front expressed by , and the approximate Pareto Optimal Front obtained by the algorithm, which we use to represent.Then the definition of the IGD can be described as：
It is worth noting that the definition of IGD in this paper is slightly different from the original definition. The main difference is the parameter in Equation (3), which we call the configuration of the benchmark functions. The configurations used in this experiment are shown in Table I. IGD compares the ideal POF with the POF obtained by other algorithms. If the distance between P and P* is closer, the smaller the value of IGD, the higher the performance of the algorithm is to some extent.
One variant of the IGD, called MIGD, can also be used to evaluate dynamic multiobjective optimization algorithms [21, 4]. MIGD is the average value of IGD values at a certain time step in each run, described as:
Where represents the set of all moments. At time t, represents the point set of the ideal POF, and represents the approximate POF obtained by the algorithm. In addition, we also hope to evaluate these algorithms in a dynamic environment, so we have defined a new indicator DMIGD based on MIGD. The definition of DMIGD is as follows:
Where is the number of different environments experienced. We used eight different environmental configurations to conduct our experiments. It is worth noting that DMIGD allows us to evaluate the dynamic multi-objective optimization algorithm from a high-level perspective, and MIGD only considers the dynamics in an environment, so it is obviously different from MIGD.
Iv-B Test Instances and Experimental Settings
In this research, we take the IEEE CEC 2015 benchmark problems set as test functions and the problem set has eleven testing functions. Details of the functions definitions are given in . In the definitions, the decision variables are and , where , and are the severity of change, maximum number of iterations, and frequency of change respectively. Table I describes the different combinations of , , and used in our experiments. Please note that, for each - combination, there will be environment changes. In other words, in all of our experiments, every test function requires 20 environmental transformations for each pair of - combination.
The POFs of the testing functions have different shapes and each function belongs to a certain DMOPs type. For example, the POF of dMOP3 and DIMP2 is convex while the POF of HE2 is discontinuous. and for FDA5,the spread of POF solutions changes over time. Table II describes the types of the testing functions. Type I means POS changes, but POF does not; Type II implies that when POS changes, POF changes accordingly; Type III indicates that POF has changed, but POS has not.
As depicted in Table II, the dimensions of the decision variables are 10 and 12, In all experiments, the population size was set to 200. As mentioned earlier, for each environment configuration, we changed each benchmark function 20 times, and in every change, we let the entire population perform 50 iterations in the evolutionary algorithm. For the incremental Support Vector Machines, we set the kernel type is Gaussian kernel and kernel scale is obtained by the Grid Search method .
Iv-C Experimental Results
In this study, we conducted two different kinds of experiments. In the first kind of experiment, we incorporate the proposed approach into the development of three well-known evolutionary algorithms, multiojective particle swarm optimization (MOPSO), nondominated sorting genetic algorithm II (NSGA-II), and the regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA). We employ six benchmark functions to test these algorithms. The experimental results are recored in Table. III. In almost all experiments, the proposed method has been significantly improved compared with the original method.
is an estimation algorithm, in which the dependence between decision variables and target variables is obtained by using a multidimensional Bayesian network. MOEA\D-KF was presented in and this idea for solving DMOPs is to predict the decision space by Kalman filter. SVM-NSGA-II was proposed in  and it is a support vector machine based dynamic MOPs algorithm.
For incremental SVM, a major iteration corresponds to contain a new example. Therefore, the computational complexity estimates must be multiplied by the number of training instances learned so far. The actual runtime depends on the balance between arithmetic operations and memory access in a small iteration. In our proposed method, when adding new samples incrementally, we can use the SVM classifier that has been obtained in the past moment to simplify the computation cost of searching the solution of the next quadratic programming. By the way, for samples that are not partitioned into the set of margin support vectors, there is no need to carry out new iterations and update core parameters, which is the key module for computation cost in incremental SVM usage.
Evolutionary algorithms can solve dynamic multi-objective optimization problems more effectively through good initial populations, but getting a good initial population is a difficult and resource-consuming problem. In order to avoid wasting valuable computing resources on repeated computing, our basic idea is that incremental training support vector machine classifiers can be trained by using the POS have been obtained at different times, and when changes occur, the parameter of SVM classifier will be updated in an online manner, then the trained classifier will filter the solution at the next moment, and the good individuals will be selected as the initial population. This initial population can help any kind of population-based algorithms to solve the dynamic multi-objective optimization problem quickly and more accurately.
This research is just a new starting point, after which we will learn how to combine incremental support vector machines with transfer learning or imbalanced learning techniques 
. On the other hand, we also want to study use deep learning methods to automatically generate better samples to solve the real world problems, such as motion generation of Multi-Legged Robot and dynamic path planning .
This work was supported by the National Natural Science Foundation of China (No.61673328).
-  M. Farina, K. Deb, and P. Amato, “Dynamic multiobjective optimization problems: test cases approximations, and applications,” IEEE Transactions on Evolutionary Computation, vol. 8, no. 5, pp. 425–442, oct 2004.
-  C. Cruz, J. R. González, and D. A. Pelta, “Optimization in dynamic environments: a survey on problems methods and measures,” Soft Comput, vol. 15, no. 7, pp. 1427–1448, dec 2010.
-  T. T. Nguyen, S. Yang, and J. Branke, “Evolutionary dynamic optimization: A survey of the state of the art,” Swarm and Evolutionary Computation, vol. 6, pp. 1–24, oct 2012.
-  A. Muruganantham, K. Tan, and P. Vadakkepat, “Evolutionary Dynamic Multiobjective Optimization Via Kalman Filter Prediction,” IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 2862–2873, Dec 2016.
-  P. Laskov, C. Gehl, S. Kr ger, and K. R. M ller, “Incremental support vector learning: analysis, implementation and applications,” Journal of Machine Learning Research, vol. 7, no. 3, p. 2006, 2006.
-  T. Tušar and B. Filipič, “Differential evolution versus genetic algorithms in multiobjective optimization,” in International Conference on Evolutionary Multi-Criterion Optimization. Springer, 2007, pp. 257–271.
B. E. Boser, I. M. Guyon, and V. N. Vapnik, “A training algorithm for optimal
margin classifiers,” in
Proceedings of the fifth annual workshop on Computational learning theory. ACM, 1992, pp. 144–152.
-  E. Osuna, R. Freund, and F. Girosi, “An improved training algorithm for support vector machines,” in Neural Networks for Signal Processing VII-Proceedings of the 1997 IEEE Workshop, 1997, pp. 276–285.
-  G. Cauwenberghs and T. Poggio, “Incremental and decremental support vector machine learning,” in International Conference on Neural Information Processing Systems, 2000, pp. 388–394.
-  C. P. Diehl and G. Cauwenberghs, “Svm incremental learning, adaptation and optimization,” in International Joint Conference on Neural Networks, 2003, pp. 2685–2690 vol.4.
-  K. Deb, S. Karthik et al., “Dynamic multi-objective optimization and decision-making using modified nsga-ii: a case study on hydro-thermal power scheduling,” in International Conference on Evolutionary Multi-Criterion Optimization. Springer, 2007, pp. 803–817.
-  H. Chen, M. Li, and X. Chen, “Using diversity as an additional-objective in dynamic multi-objective optimization algorithms,” 2009.
-  E. Alba, “Parallel evolutionary algorithms can achieve super-linear performance,” Information Processing Letters, vol. 82, no. 1, pp. 7–13, 2002.
-  M. C mara, J. Ortega, and F. D. Toro, “A single front genetic algorithm for parallel multi-objective optimization in dynamic environments,” Neurocomputing, vol. 72, no. 16, pp. 3570–3579, 2009.
-  R. Azzouz, S. Bechikh, and L. B. Said, “A dynamic multi-objective evolutionary algorithm using a change severity-based adaptive population management strategy,” Soft Computing, vol. 21, no. 4, pp. 1–22, 2015.
-  Z. Li, H. Chen, Z. Xie, C. Chen, and A. Sallam, “Dynamic multiobjective optimization algorithm based on average distance linear prediction model,” The Scientific World Journal, vol. 2014, 2014.
-  H. Guo, Y. Li, J. Shang, M. Gu, Y. Huang, and B. Gong, “Learning from class-imbalanced data: Review of methods and applications,” Expert Systems with Applications, vol. 73, pp. 220–239, 2016.
-  C. C. Coello and M. S. Lechuga, “Mopso: A proposal for multiple objective particle swarm optimization,” in Evolutionary Computation, 2002. CEC’02. Proceedings of the 2002 Congress on, vol. 2. IEEE, 2002, pp. 1051–1056.
-  Q. Zhang, A. Zhou, and Y. Jin, “Rm-meda: A regularity model-based multiobjective estimation of distribution algorithm,” IEEE Transactions on Evolutionary Computation, vol. 12, no. 1, pp. 41–63, 2008.
-  M. R. Sierra and C. A. C. Coello, “Improving pso-based multi-objective optimization using crowding, mutation and -dominance,” in International Conference on Evolutionary Multi-Criterion Optimization. Springer, 2005, pp. 505–519.
-  A. Muruganantham, K. C. Tan, and P. Vadakkepat, “Solving the IEEE CEC 2015 dynamic benchmark problems using Kalman filter based dynamic multiobjective evolutionary algorithm,” in Proceedings in Adaptation Learning and Optimization. Springer Science mathplus Business Media, nov 2015, pp. 239–252.
-  M. Helbig and A. Engelbrecht, “Benchmark functions for cec 2015 special session and competition on dynamic multi-objective optimization,” Tech. Rep., 2015.
-  S. Fine and K. Scheinberg, “Incremental learning and selective sampling via parametric optimization framework for svm,” in Advances in neural information processing systems, 2002, pp. 705–711.
-  H. Karshenas, R. Santana, C. Bielza, and P. Larranaga, “Multiobjective estimation of distribution algorithm based on joint modeling of objectives and variables,” IEEE Transactions on Evolutionary Computation, vol. 18, no. 4, pp. 519–542, 2014.
-  M. Jiang, W. Hu, L. Qiu, M. Shi, and K. C. Tan, “Solving dynamic multi-objective optimization problems via support vector machine,” in 2018 Tenth International Conference on Advanced Computational Intelligence (ICACI), March 2018, pp. 819–824.
-  M. Jiang, W. Huang, Z. Huang, and G. G. Yen, “Integration of global and local metrics for domain adaptation learning via dimensionality reduction,” IEEE Transactions on Cybernetics, vol. 47, no. 1, pp. 38–51, Jan 2017.
-  M. Jiang, Z. Huang, L. Qiu, W. Huang, and G. G. Yen, “Transfer learning-based dynamic multiobjective optimization algorithms,” IEEE Transactions on Evolutionary Computation, vol. 22, no. 4, pp. 501–514, Aug 2018.
M. Jiang, Y. Ding, B. Goertzel, Z. Huang, C. Zhou, and F. Chao, “Improving machine vision via incorporating expectation-maximization into deep spatio-temporal learning,” in2014 International Joint Conference on Neural Networks (IJCNN). IEEE, 2014, pp. 1804–1811.
-  M. Jiang, Z. Huang, G. Jiang, M. Shi, and X. Zeng, “Motion generation of multi-legged robot in complex terrains by using estimation of distribution algorithm,” in 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 2017, pp. 1–6.
-  M. Jiang, Y. Yu, X. Liu, F. Zhang, and Q. Hong, “Fuzzy neural network based dynamic path planning,” in 2012 International Conference on Machine Learning and Cybernetics, vol. 1. IEEE, 2012, pp. 326–330.