Coronavirus Optimization Algorithm: A bioinspired metaheuristic based on the COVID-19 propagation model

03/30/2020 ∙ by F. Martínez-Álvarez, et al. ∙ Universidad Pablo de Olavide 0

A novel bioinspired metaheuristic is proposed in this work, simulating how the Coronavirus spreads and infects healthy people. From an initial individual (the patient zero), the coronavirus infects new patients at known rates, creating new populations of infected people. Every individual can either die or infect and, afterwards, be sent to the recovered population. Relevant terms such as re-infection probability, super-spreading rate or traveling rate are introduced in the model in order to simulate as accurately as possible the coronavirus activity. The Coronavirus Optimization Algorithm has two major advantages compared to other similar strategies. First, the input parameters are already set according to the disease statistics, preventing researchers from initializing them with arbitrary values. Second, the approach has the ability of ending after several iterations, without setting this value either. Infected population initially grows at an exponential rate but after some iterations, the high number recovered and dead people starts decreasing the number of infected people in new iterations. As application case, it has been used to train a deep learning model for electricity load forecasting, showing quite remarkable results after few iterations.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 13

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The coronavirus (COVID-19) is a new respiratory virus, firstly reported in the city of Wuhan, China, has spread worldwide, having been reported more than 100000 infected people so far. Much remains unknown about the virus, including how many people may have very mild or asymptomatic infections, and whether they can transmit the virus. The precise dimensions of the outbreak are hard to know.

Bioinspired models typically mimic behaviors from the nature and are known for their successful application in hybrid approaches. Viruses can infect people and these people can either die, infect other people or simply get recovered after the disease. Vaccines and the immune defense system typically fight the disease and help to mitigate their effects whereas an individual is still infected. This behavior can be modeled to search for suboptimal solutions in large search spaces.

The Virus Optimization Algorithm (VOA) was proposed by Liang and Cuevas-Juárez in 2016 [13] and later improved in [14]. However, as many other metaheuristics, the results of its application are highly dependent on its initial configuration. Additionally, it simulates generic viruses, without adding individualized properties for particular viruses.

It is known that metaheuristics must deal with huge search spaces, even infinite for the continuous cases, and must find suboptimal solutions in reasonable execution time. The rapid propagation of the coronavirus along with its ability of infecting most of the countries in the world impressively fast, has inspired the novel metaheuristic proposed in this work, named Coronavirus Optimization Algorithm (CVOA). The main CVOA advantages regarding other similar approaches can be summarized as follows:

  1. Coronavirus statistics are known by the scientific community. In this sense, the rate of infection, the mortality rate or the re-infection probability are already known. That is, CVOA is parametrized with actual values for rates and probabilities, preventing the user to perform an additional study on the most suitable setup configuration.

  2. CVOA can stop the solutions exploration after several iterations, with no need to be configured. That is, the number of infected people increases during the first iterations, however, after a certain number of iterations, the number of infected people starts decreasing, until reaching a void infected set of individuals.

  3. Another relevant contribution of this work is the proposal of a new codification, discrete and of dynamic length, specifically designed for hybridizing LSTM with CVOA (or any other metaheuristic).

As for the limitations of the current approach, there is mainly one. Since there is no vaccine currently, it has not been included in the procedure to reduce the number of individuals candidates to be infected. This fact involves an exponential increase of the infected population in the first iterations and, therefore, an exponential increase of the execution time for such iterations. This fact, however, is partially solved with the isolation condition that simulates, somehow, individuals that cannot be infected at a particular iteration.

A study case is included in this work to discuss the CVOA performance. CVOA has been used to configure a Long Short-Term Memory (LSTM) architecture

[12]

, which is widely used for artificial recurrent neural network (RNN) in the field of deep learning

[5]

. Data from the Spanish electricity consumption have been used to validate the accuracy. The results achieved verge on 0.45%, substantially outperforming other well-established methods such as random forest, gradient-boost trees, linear regression or deep learning optimized with other algorithms. The code, developed in Phyton with a discrete codification, is available in the supplementary material (along with an academic version in Java for a binary codification).

Finally, the authors acknowledge the need of further study on the performance of well-known functions [11], however, given the relevance of coronavirus is acquiring throughout the world (declared as pandemic by the World Health Organization) and the remarkable results achieved when combined with deep learning, they wanted to share this preliminary work hoping it inspires future research in this direction.

The rest of the paper is organized as follows. Section 2 discusses related and recent works. The methodology proposed is introduced in 3. Section 4 proposes a discrete codification to hybridize deep learning models with CVOA and provides some illustrative cases. The results achieved are reported and discussed in 6. Finally, the conclusions drawn and future work suggestions are included in Section 7.

2 Related works

There are many bioinspired metaheuristics to solve optimization problems. Although CVOA has been conceived to optimize any kind of problems, this section focuses on optimization algorithms applied to hybridize deep learning models.

It is hard to find consensus among the researchers on which method should be applied to which problem, and, for this reason, many optimization methods have been proposed during the last decade to improve deep learning models. Generally, the criterion for selecting a method is its associated performance from a wide variety of perspectives. Low computation cost, accuracy or even implementation difficulty can be accepted as one of these criteria.

One of the most extended metaheuristics used to improve deep learning parameters is genetic algorithms (GA). Hence, a LSTM network optimized with GA can be found in

[4]. To evaluate the proposed hybrid approach, the daily Korea Stock Price Index data were used, outperforming the benchmark model. In 2019, a network traffic prediction model based on LSTM and GA was proposed in [3]. The results were compared to pure LSTM and ARIMA, reporting higher accuracy.

Multi-agents systems have also been applied to optimize deep learning models. The use of Particle Swarm Optimization (PSO) can be found in

[15]

. The authors proposed a model based on kernel principal component analysis and back propagation neural network with PSO for midterm power load forecasting. The hybridization of deep learning models with PSO was also explored in

[8] but, this time, the authors applied the methodology with image classification purposes.

Ants colony optimization (ACO) models have also been used to hybridize deep learning. Thus, Desell et al. [6] proposed an evolving deep recurrent neural networks using ACO applied to the challenging task of predicting general aviation flight data. The work in [7] introduced a method based on ACO to optimize a LSTM recurrent neural networks. Again, the field of application was flight data records obtained from an airline containing flights that suffered from excessive vibration.

Some papers exploring the Cuckoo Search (CS) properties have been published recently as well. In [18]

, CS was used to find suitable heuristics for adjusting the hyper-parameters of another LSTM network. The authors claimed an accuracy superior to 96% for all the datasets examined. Nawi et al.

[17] proposed the use of CS to improve the training of RNN in order to achieve fast convergence and high accuracy. Results obtained outperformed those than other metaheuristics.

The use of the artificial bee colony (ABC) optimization algorithm applied to LSTM can also be found in the literature. Hence, and optimized LSTM with ABC to forecast the bitcoin price was introduced in [21]. The combination of ABC and RNN was also proposed in [2]

for traffic volume forecasting. This time the results were compared to standard backpropagation models.

From the analysis of these works, it can be concluded that there is an increasing interest in using metaheuristics in LSTM models. However, not as many works as for artificial neural networks can be foundin the literature and, none of them, based on a virus propagation model. These two facts, among others, justify the application of CVOA to optimize LSTM models.

3 Methodology

This section introduces the CVOA methodology. Thus, Section 3.1 describes the steps. Section 3.2 suggests how parameters must be set. Section 3.3 shows the CVOA pseudocode. Finally, Section 3.4 comments the pseudocode.

3.1 Steps

Step 1. Generation of the initial population. The initial population consists of one individual, the so-called patient-zero (). As in the coronavirus epidemy, it identifies the first human being infected.

Step 2. Disease propagation. Depending on the individual, several cases are evaluated:

  1. Some of the infected individuals die. They die according to the coronavirus death rate (). For simplicity, it is considered that such individuals cannot infect new individuals.

  2. The individuals surviving the coronavirus will infect new individuals (intensification). Two types of spreading are considered, according to a biven probability ():

    • Ordinary spreaders. Infected individuals will infect new ones according to the coronavirus spreading rate ().

    • Super-spreaders. Infected individuals will infect new ones according to the coronavirus superspreading rate ().

  3. There is another consideration, since it is needed to ensure diversification. Both ordinary and super-spreaders individuals can travel and explore solutions quite dissimilar. Therefore, individuals have a probability of traveling () thus allowing to propagate the disease to solutions that may be quite different (). In case of not being traveler, new solutions will change according to an . One individual can be both super-spreader and traveler.

Step 3. Updating populations. Three main populations are maintained and updated for each generation.

  1. Dead population. If any individual dies, it is added to this population and can never be used again.

  2. Recovered population. After each iteration, infected individuals (after spreading the coronavirus according to the previous step) are sent to the recovered population. It is known that there is a reinfection probability. Hence, an individual belonging to this population could be re-infected at any iteration provided that it meets the reinfection criterion (). Another situation has to be considered as well, since individuals can be isolated, simulating the how the population has behaved according to local governments policies. For the sake of simplicity, it is considered that an isolated individual is sent to the recovered population as well when meeting an isolation probability ().

  3. New infected population. This population gathers all individuals infected at each iteration, according the procedure described in the previous steps.

Step 4. Stop criterion. One of the most interesting features of the proposed approach lies on its ability to end without the need of controlling any parameter. This situation occurs because the recovered and dead populations are constantly growing as time goes by, and the new infected population cannot infect new individuals. It is expected that the number of infected individuals increases for a certain number of iterations. However, from a particular iteration on, the size of the new infected population will be smaller than that of the current one because recovered and dead populations are too big, and the size of the infected population decays over time. Additionally, a preset number of iterations () can be added to the stop criterion.

3.2 Suggested parameters setup

Since CVOA simulates the coronavirus disease propagation, most of the rates (propagation, re-infection or death) are already known. This fact prevents the research from wasting time in selecting values for such rates and turns the CVOA into metaheuristic quite easy to execute. The suggested values and associated discussion are listed below:

  1. . An infected individual can die with a given probability. Currently, this rate is set as almost 5% by the scientific community. Therefore, .

  2. . It is the probability that an individual spread the disease to a greater number of healthy individuals. It is known that this situation affects to a 10% of the population, therefore, . After this condition is validated, two situations can be found:

    • . If the infected individual is not a super-spreader, then the infection rate is 2.5. It is suggested that this rate varies from 0 to 5.

    • . If the infected individual turns out to be super-spreader, then he/she infects up to 15 healthy individuals, as per reported by the scientific community. It is suggested that this rate varies from 6 to 15.

  3. . It is known that a recovered individual can be re-infected. The current reported rate is 14%. Therefore, .

  4. . This value is uncertain because countries are taking different policies to isolate people. This parameter helps to reduce the exponential growth of the infected population after each iteration. Therefore a high value must be assigned to. It is suggested that

  5. . This probability simulates how an infected individual can travel to any place in the world and can infect any healthy individual. It is known that almost a 10% of the population travel during a week (simulated time for every iteration), so .

  6. . This parameter simulates the duration of the pandemic. Since the estimated recovering time is one week, each iteration simulates one week. Currently, this data is unknown so this number can be adjusted to the size of the problem. It is suggested that

    .

3.3 Pseudo codes

This section provides the pseudo code of the most relevant functions for the CVOA, along with some comments to better understand them.

3.3.1 Function

This is the main function and its pseudo code can be found in Algorithm 1. Four lists must be maintained: dead, recovered, infected (the current set of infected individuals) and new infected individuals (the set of new infected individuals, generated by the spreading of the coronavirus from the current infected individuals).

The initial population is generated by means of the patient zero (), which is a random solution.

The number of iterations is controlled by the main loop, evaluating the duration of the pandemic (preset value) and if there is still any infected individual. In this loop, every individual can either die (it is sent to the dead list) or infect, thus enlarging the size of the new infected population. How this infection is made, is implemented in the function (see Section 3.3.2).

Once the new population is formed, it is evaluated any solution outperforming the current one has been found and, in such case, the best individual is updated.

1:  define infectedPopulation, newInfectedPopulation as of
2:  define dead, recovered as of
3:  define PZ, bestIndividual, currentBestIndividual, aux as
4:  define time as
5:  define bestSolutionFitness, currentbestFitness as
6:  time 0
7:  PZ InfectPatientZero()
8:  infectedPopulation PZ
9:  bestIndividual PZ
10:  while time sizeof(infectedPopulation) 0  do
11:     dead die(infectedPopulation)
12:     for all  do
13:        aux infect(i,recovered,dead)
14:        if notnull(aux) then
15:           newInfectedPopulation aux
16:        end if
17:     end for
18:     currentBestIndividual selectBestIndividual(newInfectedPopulation)
19:     if fitness(currentBestIndividual) bestIndividual then
20:        bestIndividual currentBestIndividual
21:     end if
22:     recovered infectedPopulation
23:     clear(infectedPopulation)
24:     infectedPopulation newInfectedPopulation
25:     time time 1
26:  end while
27:  return  bestIndividual
Algorithm 1 Function

3.3.2 Function

This function receives an infected individual and returns the set of new infected individuals. Two additional lists, recovered and dead, are also received as input parameters since they must be updated after the evaluation of every infected individuals. The pseudo code is shown in Algorithm 2

Two conditions are evaluated to determine the number of new infected individuals (use of or ) or how different the new individuals will be ( or . The implementation on how this new infected individuals are encoded according to such rates is carried out in the function .

0:  infected as of ; recovered, dead as of
1:  define R1, R2 as
2:  define newInfected as of
3:  R1 RandomNumber()
4:  R2 RandomNumber()
5:  if R1  then
6:     if R2  then
7:        newInfected newInfection (infected, recovered, dead, , )
8:     else
9:        newInfected newInfection (infected, recovered, dead, , )
10:     end if
11:  else
12:     if R2  then
13:        newInfected newInfection (infected, recovered, dead, , )
14:     else
15:        newInfected newInfection (infected, recovered, dead, , )
16:     end if
17:  end if
18:  return  newInfected
Algorithm 2 Function

3.3.3 Function

Given an infected individual this function generates new infected individuals, according to the spreading and traveling rates. This function also controls that the new infected individuals are not already in the dead list (in such case this new infection is ignored) or in the recovered list (in such case the is applied to determine whether the individual is re-infected or if it remains in the recovered list). Additionally, it considers that the new potential infected individual might be isolated, which is controlled by . Although the use of an extra list could be implemented, it has been decided to treat these individuals as recovered. Therefore, if an isolated individual is attempted to be infected, it is added to the recovered list.

The effective generation of the new infected individuals must be carried in the function , whose pseudo code is not provided because it depends on the codification and the nature of the problem to be optimized. This function must return a set of new infected individuals, according to the aforementioned rates. Specific information on how this codification and replication is done for LSTM models.

The pseudo code for the described procedure can be found in Algorithm 3.

0:  infected as ; recovered, dead as list of
1:  define R3, R4 as
2:  define newInfected as of
3:  R3 RandomNumber()
4:  R4 RandomNumber()
5:  aux replicate(infected, , )
6:  for all  do
7:     if i dead then
8:        if i recovered then
9:           if  then
10:              newInfected i
11:           else
12:              
13:           end if
14:        else if  then
15:           newInfected i
16:           remove i from recovered
17:        end if
18:     end if
19:  end for
20:  return  newInfected
Algorithm 3 Function

3.3.4 Function

This function is called from the function. It evaluates all individuals in the infected population and determines whether they die or not, according to the given . Those meeting this condition, are sent to the dead list. Algorithm describes this procedure.

0:  infectedPopulation as of
1:  define dead as of
2:  define R5 as
3:  for all i infectedPopulation do
4:     R5 RandomNumber()
5:     if R5  then
6:        dead i
7:     end if
8:  end for
9:  return  dead
Algorithm 4 Function

3.3.5 Function

This is an auxiliary function used to find the best fitness in a list of infected individuals. Its peudo code is shown in Algorithm 5.

0:  infectedPopulation as of
1:  define bestIndividual as
2:  define bestFitness as
3:  bestFitness
4:  for all i infectedPopulation do
5:     if fitness(i) bestFitness then
6:        bestFitness fitness(i)
7:        bestIndividual i
8:     end if
9:  end for
10:  return  bestIndividual
Algorithm 5 Function

4 Hybridizing deep learning with CVOA

In order to apply CVOA to optimize the architecture of a deep neural network, the individual codification shown in Figure 1 has been implemented.

Figure 1: Individual codification for hybridizing deep learning architectures using the proposed CVOA algorithm.

As it can be seen in Figure 1, each individual is composed of the following elements. The element LR encodes the learning rate used in the neural network algorithm. It can take a value from to and its corresponding decoded values are , , , , and .

The element DROP encodes the dropout rate applied to the neural network. It can take values from to that correspond to , , , , , , , and , respectively. The dropout rate is distributed uniformly for all the layers of the network. That is, if the dropout is and the network has layers, then the (

) of the neurons of each layer will be removed.

The element of the individual stores the number of layers of the network. It is restricted to . The first layer is referred to the input layer of the neural network. The rest of layers are hidden layers. The output layer is excluded from the codification. Therefore, the optimized network can contain from to hidden layers.

The proposed individual codification has a variable size. Thus, its size depends on the number of layers indicated in the element . Consequently, a list of elements (LAYER , …, LAYER ) are also included in the individual, which encode the number of units (neurons) for each network layer. Each of these elements can take values from to , and their corresponding decoded values range from to , with a step of .

4.1 PZ generation

The PZ, as it has been described previously, is the individual of the first iteration in the CVOA algorithm. Following the particular hybridization proposed, a random individual is created taking into account the codification defined above.

In first place, a random value for the learning rate of the PZ is generated. Specifically, a number between and

is generated randomly in an uniform distribution. Such limits are the indicated in Figure

1, according to the possible encoded values of the learning rate element. The same process is carried out to produce a random value for the dropout element. In such case, a random number between and is generated.

In second place, a random number of layers is generated for the element of the patient zero individual. Such number layers is a random number between and . Note that the first layer is reserved for the input layer of the neural network, as it has been discussed before.

In last place, for each one of the layers, a random number of units is generated between and , covering the possible encoded values for the number of units previously defined (see Figure 1).

4.2 Infection procedure

The infection procedure described here corresponds to the functionality of , introduced in the line of the Algorithm 3. This procedure takes an individual as input and returns an infected individual according to the following procedure.

The first step is to determine the element of the infected individual that will be mutated. The probability of such mutation occurs has been set to so that every element has the same probability to mutate. If the mutation occurs, then the element of the individual is modified according to the process described in Section 4.4.

If the element (the number of layers of the network) changes, then the elements encoding the different layers within the individual (LAYER , …, LAYER ) must be resized accordingly. Such resizing process is explained in Section 4.3.

The second step is to determine how many elements of the individual will be infected. If the , then the number of infected elements is generated randomly from to the length of the individual (excluding the element ). Else, the indicates itself the number of infected elements.

As third step, once it is determined the number of infected elements of the individual, a list of random positions are generated. For example, if three positions of the individual must be changed, then the random positions affected could be, for instance, whose referred to the elements {DROP, LAYER , LAYER }.

Finally, the selected positions of the individual are mutated. Such mutation is described in Section 4.4.

4.3 Individual resizing process

When an individual is infected at the position of the element , the list of elements that encodes the number of units per layer (LAYER , …, LAYER ) must be resized accordingly.

In the case that the new number of layers after the infection is lower than its previous value, then the last leftover elements are removed. For instance, if the initial individual is (four layers), the element is infected and the new value is , then the resulting individual will be .

In the case that the new number of layers after the infection is higher than its previous value, the new random elements are added at the end of the individual. For instance, if the initial individual is (four layers), the element is infected and the new value is , then the resulting individual could be .

4.4 Single position mutation

The process carried out to change the value of a specific element of an individual is described below.

First, a signed change amount is randomly determined using the following criteria. A random real number between and is generated using an uniform distribution. If , then the change amount will be . Else if , then the change amount will be . Else if , then the change amount will be . Else, the change amount will be .

Once the amount of change is determined, the new value for the infected element is computed. If its previous value is , then the new value after the single position mutation will be . If the new value exceeds the limits defined for the individual codification, such value is set to the maximum or minimum allowed value accordingly.

5 CVOA preliminary analysis

This section provides an overview on how populations evolve over time, and how the search space is explored to reach the optimum value for a given fitness function.

To conduct this experimentation, a simple binary codification has been used. The fitness function was because to goal of this section is to evaluate the growth of the new infected populations, and to the find challenging optimum values. This function reaches the minimum value at , that is, .

For this reason, individuals with 10, 20, 30, 40 and 50 bits have been tested. Tables 1-5 summarize the results achieved for each of these lengths, respectively. Every experiment has been launched 50 times, determining that, on average, the optimum value was found for 11, 12, 14, 15 and 17 iterations, respectively. Each table shows the results of a particular execution meeting this criterion.

Iteration Deaths Recovered Infected Fitness
1 0 1 6 7.72E+04
2 1 7 8 6.05E+04
3 1 15 5 6,00E+04
4 1 20 6 4
5 2 26 7 4
6 2 33 5 4
7 2 38 19 4
8 3 48 13 1
9 3 61 16 1
10 4 77 18 1
11 5 95 20 0
Table 1: Sample execution for a 10-bit binary codification.
Iteration Deaths Recovered Infected Fitness
1 0 1 10 5.73E+11
2 1 11 14 3.92E+11
3 2 25 16 3.14E+11
4 2 41 24 1.33E+09
5 4 63 36 1.37E+07
6 5 98 56 2.76E+06
7 8 153 72 1.32E+06
8 12 222 112 1.61E+04
9 17 330 172 1
10 26 499 249 1
11 38 738 389 1
12 58 1112 567 0
Table 2: Sample execution for a 20-bit binary codification.
Iteration Deaths Recovered Infected Fitness
1 0 1 8 6.11E+17
2 1 9 9 6.00E+16
3 1 18 18 5.59E+16
4 2 36 21 1.12E+16
5 3 57 43 1.04E+16
6 5 100 83 1.85E+13
7 9 183 126 1.08E+10
8 16 307 208 1.51E+09
9 26 514 326 1.21E+09
10 42 838 606 1.07E+09
11 73 1436 1049 1.69E+07
12 125 2467 1796 676
13 215 4233 2967 9
14 363 7138 4823 0
Table 3: Sample execution for a 30-bit binary codification.
Iteration Deaths Recovered Infected Fitness
1 0 1 10 2.23E+18
2 1 11 20 2.57E+15
3 2 31 41 1.83E+11
4 4 70 85 3.58E+09
5 8 154 170 3.24E+09
6 17 321 331 7.39E+08
7 33 652 598 7.08E+08
8 63 1246 1036 1.06E+08
9 115 2275 1918 2.12E+07
10 211 4169 3592 1612900
11 390 7716 6357 270400
12 708 14003 11205 2809
13 1268 25052 19495 49
14 2243 44198 33215 9
15 3904 76672 55122 0
Table 4: Sample execution for a 40-bit binary codification.
Iteration Deaths Recovered Infected Fitness
1 0 1 12 4.61E+18
2 1 13 23 7.80E+15
3 2 36 23 3.62E+13
4 2 36 40 1.39E+12
5 4 76 71 1.37E+12
6 8 146 119 1.57E+10
7 14 265 185 7.50E+08
8 23 449 338 1.21E+08
9 40 787 586 7963684
10 69 1369 1100 7873636
11 124 2461 2129 597529
12 230 4579 3957 68121
13 428 8499 7211 17956
14 789 15644 13305 36
15 1454 28807 24167 36
16 2662 52622 43184 1
17 4821 95116 76288 0
Table 5: Sample execution for a 50-bit binary codification.

Table 6 summarizes the amount of search space explored, on average, before finding the optimum value. It can be seen that for a small space as possible values, the optimum value is reached after exploring 15.6250% valid solutions. However, this value acutely decreases as the search space increases. The opposite case is reached for possible values, where the optimum value was reached after exploring just 0.00000002% of valid solutions.

Length Search space Evaluated Evaluated (%)
10 1024 160 15.62500000%
20 1048576 2253 0.21486282%
30 1073741824 15589 0.00145184%
40 1,09951E+12 170827 0.00001554%
50 1,1259E+15 219613 0.00000002%
Table 6: Search space explored for a binary codification with different individuals length.

Figure 2 illustrates how the new infected population evolve over time for the 20-bit codification case with the suggested parameters initialization. It can be seen that the number of new people increase until a given iteration in which it starts to decrease. Note that data shown in the figure are sampled every 5 iterations.

Figure 2: Number of new infected individuals for a 20-bit binary codification execution.

Finally, Figure 3 shows the number of recovered and dead people. These two curves accumulates these numbers since dead and recovered people are sent to their respective lists and are no longer infected (except for those in recovered that can be reinfected for a its given probability ). The final number of recovered is 92826 and of dead people 5335, being 94.56% and 5.44%, respectively.

Figure 3: Total number of recover and dead people for a 20-bit binary codification execution.

6 Results

This section reports the results achieved by hybridizing a deep learning model with CVOA. Section 6.1 describes the study case selected to prove the effectiveness of the proposed algorithm. Section 6.2 describes the dataset used. Section 6.3 discusses the results achieved and includes some comparative methods.

6.1 Study case: electricity demand time series forecasting

The forecasting of future values fascinates the human being. To be able to understand how certain variables evolve over time has many benefits in many fields.

Electricity demand forecasting is not an exception, since there is a real need for planning the amount to be generated or, in some countries, to be bought.

The use of machine learning to forecast such time series has been intensive during the last years

[16]. But, with the development of deep learning models, and, in particular of LSTM, much research is being conducted in this application field [1].

6.2 Dataset description

The time series considered in this study is related to the electricity consumption in Spain from January 2007 to June 2016, the same as used in [19]. It is a time series composed of 9 years and 6 months with a high sampling frequency (10 minutes), resulting in 497832 measures in total into a 33 MB file.

As in the original paper, the prediction horizon is 24, that is, this is a multi-step strategy with . The size of samples used for the prediction of these 24 values is 168. Furthermore, the dataset was split into 70% for the training set and 30% for the test set, and in addition, a 30% of the training set has also been selected for the validation set, in order to find the optimal parameters. The training set covers the period from January 1, 2007 at 00:00 to August 20, 2013 at 02:40. Therefore, the test set comprises the period from August 20, 2013 at 02:50 to June 21, 2016 at 23:40.

6.3 Performance analysis

This section reports the results obtained by applying hybridizing LSTM with CVOA, by means of the codification proposed in Section 4, to forecast the Spanish electricity dataset described in Section 6.2.

Linear regression (LR), decision tree (DT), gradient-boosted trees (GBT) and random forest (RF) models have been used with a parametrization setup according to that studied in

[10, 9]. A deep neural network optimized with a grid search (DNN-GS) according to [19] has also been applied. Finally, another deep neural network but optimized with random search (DNN-RS) and smoothed with a low-pass filter (DNN-RS-LP) [20] have also been applied.

These results along with those of LSTM-CVOA are summarized in Table 7, expressed in terms of the mean absolute percentage error (MAPE). It can be observed that LSTM-CVOA outperforms all evaluated methods which have showed particularly remarkable performance for this real-world dataset.

Another relevant consideration that must be taken into account is that the compared methods generated 24 independent models, each of them for every value forming . So it would expected that LSTM-CVOA performance increases if independent models were generated for each of the values in .

Method MAPE (%)
LR 7.34
DT 2.88
GBT 2.72
RF 2.20
DNN-GS 1.68
DNN-RS 1.57
DNN-RS-LP 1.36
LSTM - CVOA 0.47
Table 7: Results in terms of MAPE for CVOA-LSTM compared to other well established methods.

These results have been achieved with the individual , which decoded involves the following architecture parameters:

  • Learning rate: 10E-04.

  • Dropout: 0.

  • Number of layers: 8.

  • Units per layer:

7 Conclusions and future works

This work has introduced a novel bioinspired metaheuristic, based on the coronavirus behavior. One the one hand, CVOA has two major advantages. First, its highly relation to the coronavirus spreading model, prevents the authors to make any decision about the inputs values. Second, it ends after a certain number of iterations due to the exchange of individuals between healthy and dead/recovered lists. Third, a novel discrete and dynamic codification has been proposed to hybridize deep learning models. On the other hand, it exhibits some limitations. Such is the case for the exponential growth of the infected population as time (iterations) goes by. Additional experimentation must be conducted in order to assess its performance on standard functions and find out the search space shapes in which it can be more effective.

Some actions must be taken to reduce the size of the infected population after a number of iterations, that grows exponentially. In this sense, a vaccine must be implemented. This case would involve adding to the recovered list, at a given healthy individuals.

Another suggested research line is using dynamic rates. For instance, the observation of the preliminary effects of the isolation in countries like China or Italy suggests that the could be simulated as a Poisson process, but more time and country recoveries is required to confirm this trend.

A parallel and distributed version is proposed for future works. CVOA can be easily transformed into a multi-agent metaheuristic, in which different agents search for the best solution in a collaborative way. In this sense, the simulation of the existence of more strains can be considered. So far, it is known that there exists one strain way more aggressive than the original one. This could be modeled as a new agent with different initial setup (higher , for instance), sharing recovered or dead lists.

Finally, for the particular multi-step forecasting problem analyzed it would be desirable to generated independent models for each of the values that form the prediction horizon .

Supplementary material

Along with this paper, an academic version in Java for a binary codification is provided, with a simple fitness function (https://github.com/DataLabUPO/CVOA_academic). Additionally, the code in Phyton for the deep learning approach is also provided, with a more complex codification and the suggested implementation, according to the pseudocode provided (https://github.com/DataLabUPO/CVOA_LSTM).

Acknowledgments

The authors would like to thank the Spanish Ministry of Economy and Competitiveness for the support under project TIN2017-88209-C2.

References

  • [1] J. Bedi and D. Toshniwal. Deep learning framework to forecast electricity demand. Applied Energy, 238:1312–1326, 2019.
  • [2] A. Bosire. Recurrent Neural Network Training using ABC Algorithm for Traffic Volume Prediction. Informatica, 43:551–559, 2019.
  • [3] J. Chen, H. Xing, H. Yang, and L. Xu. Network Traffic Prediction Based on LSTM Networks with Genetic Algorithm. Lecture Notes in Electrical Engineering, 550:411–419, 2019.
  • [4] H. Chung and K.-S. Shin. Genetic Algorithm-Optimized Long Short-Term Memory Network for Stock Market Prediction. Sustainability, 10(10):3765, 2018.
  • [5] S. De-Cnudde, Y. Ramon, D. Martens, and F. Provost. Deep learning on big, sparse, behavioral data. Big Data, 7(4):286–307, 2019.
  • [6] T. Desell, S. Clachar, J. Higgins, and B. Wild. Evolving deep recurrent neural networks using ant colony optimization. Lecture Notes in Computer Science, 9026:86–98, 2015.
  • [7] A. ElSaid, F. ElJamiy, J. Higgings, B. Wild, B. Wild, and T. Desell. Using ant colony optimization to optimize long short-term memory recurrent neural networks. In

    Proceedings of the Genetic and Evolutionary Computation Conference

    , pages 13–20, 2018.
  • [8] F. E. Fernandes-Junior and G. G. Yen. Particle swarm optimization of deep neural networks architectures for image classification. Swarm and Evolutionary Computation, 49:62–74, 2019.
  • [9] A. Galicia, R. L. Talavera-Llames, A. Troncoso, I. Koprinska, and F. Martínez-Álvarez. Multi-step forecasting for big data time series based on ensemble learning. Knowledge-Based Systems, 163:830–841, 2019.
  • [10] A. Galicia, J. F. Torres, F. Martínez-Álvarez, and A. Troncoso. Scalable forecasting techniques applied to big electricity time series. Lecture Notes in Computer Science, 10306:165–175, 2019.
  • [11] F. Glover and G. A. Kochenberger. Handbook of metaheuristics. Springer, 2003.
  • [12] A. Kelotra and P. Pandey. Stock market prediction using optimized deep-convlstm model. Big Data, 8(1):5–24, 2020.
  • [13] Y. C. Liang and J. R. Cuevas-Juárez. A novel metaheuristic for continuous optimization problems: Virus optimization algorithm. Engineering Optimization, 48(1):73–93, 2016.
  • [14] Y. C. Liang and J. R. Cuevas-Juárez. A self-adaptive virus optimization algorithm for continuous optimization problems. Soft Computing, In press, 2020.
  • [15] Zhao Liu, Xincheng Sun, Shuai Wang, Mengjiao Pan, Yue Zhang, and Zhendong Ji. Midterm power load forecasting model based on kernel principal component analysis and back propagation neural network with particle swarm optimization. Big Data, 7(2):130–138, 2019.
  • [16] F. Martínez-Álvarez, A. Troncoso, G. Asencio-Cortés, and J. C. Riquelme. A survey on data mining techniques applied to electricity-related time series forecasting. Energies, 8(11):13162–13193, 2015.
  • [17] N. M. Nawi, A. Khan, and M. Z. Rehman. A New Optimized Cuckoo Search Recurrent Neural Network (CSRNN). In Proceedings of the International Conference on Robotic, Vision, Signal Processing & Power Applications, pages 335–341, 2014.
  • [18] D. Srivastava, Y. Singh, and A. Sahoo. Auto Tuning of RNN Hyper-parameters using Cuckoo Search Algorithm. In Proceedings of the International Conference on Contemporary Computing, pages 1–5, 2019.
  • [19] J. F. Torres, A. Galicia, A. Troncoso, and F. Martínez-Álvarez. A scalable approach based on deep learning for big data time series forecasting. Integrated Computer-Aided Engineering, 25(4):335–348, 2018.
  • [20] J. F. Torres, D. Gutiérrez-Avilés, A. Troncoso, and F. Martínez-Álvarez. Random hyper-parameter search-based deep neural network for power consumption forecasting. Lecture Notes in Computer Science, 11506:259–269, 2019.
  • [21] A. D. Yuliyono and A. S. Girsang. Artificial Bee Colony-Optimized LSTM for Bitcoin Price Prediction. Advances in Science, Technology and Engineering Systems Journal, 4(5):375–383, 2019.