Enhanced Estimation of Autoregressive Wind Power Prediction Model Using Constriction Factor Particle Swarm Optimization

08/21/2014 ∙ by Adnan Anwar, et al. ∙ UNSW Canberra UNSW 0

Accurate forecasting is important for cost-effective and efficient monitoring and control of the renewable energy based power generation. Wind based power is one of the most difficult energy to predict accurately, due to the widely varying and unpredictable nature of wind energy. Although Autoregressive (AR) techniques have been widely used to create wind power models, they have shown limited accuracy in forecasting, as well as difficulty in determining the correct parameters for an optimized AR model. In this paper, Constriction Factor Particle Swarm Optimization (CF-PSO) is employed to optimally determine the parameters of an Autoregressive (AR) model for accurate prediction of the wind power output behaviour. Appropriate lag order of the proposed model is selected based on Akaike information criterion. The performance of the proposed PSO based AR model is compared with four well-established approaches; Forward-backward approach, Geometric lattice approach, Least-squares approach and Yule-Walker approach, that are widely used for error minimization of the AR model. To validate the proposed approach, real-life wind power data of Capital Wind Farm was obtained from Australian Energy Market Operator. Experimental evaluation based on a number of different datasets demonstrate that the performance of the AR model is significantly improved compared with benchmark methods.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

In recent years, renewable energy has gained much popularity and attention because of it’s potential in economic and environmental advantages. Some of the benefits include- high stainability, low carbon emission, reduction of environmental impact, saving fuel cost and so on. Other advantages include economical benefits to remote communities and supporting the microgrids during the operation in islanded mode. Although renewable energy, e.g., wind and solar, offers huge benefits [1], their practical use is limited due to their intermittent nature which makes it very challenging to ensure a steady power supply in the grid [2, 3]. Because of the variable nature of the renewable energy based power generation sources, transmission and distribution system operators need advanced monitoring and control.

Wind power generation relies on wind speed which varies depending on location and time. For economic and stable operation of the wind power plant, accurate forecasting of wind power is critical. There are two main wind power forecasting approaches, physical method and statistical method. In the first approach, the physical system and power translation processes are modelled in detail. Therefore, physical approaches not only need the information of historical wind speed data but also other information, i.e., meteorological output, hub height of the turbine and physical modelling of power conversion process from wind speed are essential [4]. On the other hand, in a statistical approach, wind power output is modelled as a time-series where the power output at any time instant depends on its previous observation values. The physical approach provides good accuracy for long term forecasting but not so good for short term forecasting as it is computationally very demanding. On the contrary, statistical approaches are well suited for short therm forecasting.

For short term wind power forecasting, different approaches are well studied [5, 6, 7, 8, 9, 10, 11, 12]. In a conventional statistical approach, wind power output behaviour is modelled as a time-series. Autoregressive (AR) model has been used for wind energy forecasting in [8, 9] and Autoregressive Moving Average (ARMA) model has been used in [6, 7]

. The Artificial Neural Network (ANN) is also widely used 

[10, 11, 12]. However, the ANN based approaches has very slow convergence during the training phase [13]. On the other hand, statistical regressive models are computationally very efficient and widely used for short term forecasting [6, 7, 8, 9].

In the statistical approaches, the forecasting accuracy is highly dependent on the estimated model of the wind power output behaviour. Therefore, it is important to identify the estimated model parameters accurately. Different methods are widely used to estimate the AR model parameters, such as, ordinary Least Squares (LS) approach, Forward Backward (FB) approach, Geometric Lattice (GL) approach and Yule-Walker (YW) approach, etc 

[14, 15, 16]. As the wind power output has variable characteristics, the error function obtained from the estimated model may have many local minima. For short-term load forecasting, it has been shown that the Particle Swarm Optimization (PSO), one of the major paradigms of the computational swarm intelligence, converges to the global optimal solution of a complex error surface and finds better solution compared with gradient search based stochastic time-series techniques [13]. Previously, PSO has been widely used in different applications of power system [17, 18]. In this work, a modified variant of PSO based on Constriction Factor (CF) is employed to identify the AR parameters more accurately. The proposed CF-PSO based identified AR parameters have better error minimization profiles compared to the well-established LS, FB, GL and YW based approaches.

The organization of this paper is as follows- The formulation of basic PSO and CF-PSO is discussed in Section II

. Autoregressive model order selection and parameter estimation methodology is described in Section 

III. The proposed AR Parameter Estimation method based on CF-PSO is illustrated in Section IV. In Section V, results obtained from this experiment are given and compared with four standard techniques. Finally, the paper concludes with some brief remarks in Section VI.

Ii PSO Formulation: Parameters and Variants

PSO is a multi-objective optimization technique which finds the global optimum solution by searching iteratively in a large space of candidate solutions. The description of basic PSO and CF-PSO formulation is discussed in the following subsections:

Ii-a Basic PSO

This meta-heuristic is initialized by generating random population which is referred as a swarm. The dimension of the swarm depends on the problem size. In a swarm, each individual possible solution is represented as a ‘particle’. At each iteration, positions and velocities of particles are updated depending on their individual and collective behavior. Generally, objective functions are formulated for solving minimization problems; however, the duality principle can be used to search the maximum value of the objective function 

[19]. At the first step of the optimization process, an n

-dimensional initial population (swarm) and control parameters are initialized. Each particle of a swarm is associated with the position vector and the velocity vector, which can be written as

velocity vector, and

position vector,

where n represents the search space dimension. Before going to the basic PSO loop, the position and velocity of each particle is initialized. Generally, the initial position of the particle

can be obtained from uniformly distributed random vector U (

), where and represents the lower and upper limits of the solution space respectively.

During the optimization procedure, position of each particle is updated using (1)


where and .

At each iteration, new velocity for each particle is updated which drives the optimization process. The new velocity of any particle is calculated based on its previous velocity, the particle’s best known position and the swarm’s best known position. Particle’s best known position is it’s location at which the best fitness value so far has been achieved by itself and swarm’s best known position is the location at which the best fitness value so far has been achieved by any particle of the entire swarm [20]. The velocity equation drives the optimization process which is updated using (2)


In this equation, w is the inertia weight. represents the ‘self influence’ of each particle which quantifies the performance of each particle with it’s previous performances. The component represents the ‘social cognition’ among different particles within a swarm and quantify the performance relative to other neighboring particles. The learning co-efficients and represent the trade-off between the self influence part and the social cognition part of the particles [21]. The values of and are adopted from previous research and is typically set to 2 [22]. In eqn (2), is particle’s best known position and is swarm’s best known position.

In the solution loop of PSO, the algorithm continues to run iteratively, until one of the following stopping conditions is satisfied [23].

  1. Number of iterations reach the maximum limit, e.g., 100 iterations.

  2. No improvement is observed over a number of iterations, e.g., error less than  = 0.001

Ii-B Constriction factor PSO (CF-PSO) with boundary conditions

To achieve better stability and convergent behavior of PSO, a constriction factor has been introduced by Clerc and Kennedy in [24]. The superiority of CF-PSO over inertia-weight PSO is discussed in [20]

. Basically, the search procedure of CF-PSO is improved using the eigenvalue analysis and the system behavior can be controlled which ensures a convergent and efficient search procedure 

[25]. To formulate CF-PSO, (2) is replaced by (3)-(5)[18, 26, 27, 28].






here the value of must be greater than 4 to ensure a stable and convergent behavior [20, 24]. Usually, the value of is set to 4.1 (); therefore, the value of k becomes 0.7298 [26, 27, 28].

Boundary condition [29], which helps to keep the particles within allowable solution space, is also applied in this research as shown below:


Iii Autoregressive (AR) Model

AR is a univariate time-series analysis model that is widely used for model estimation and forecasting. In an AR model, the output variable has linear association with its own previous observations. For a sample period of , a -order AR model can be written following the expression below [8, 16]:


where, are the lag parameters of the model, is the constant term, and is the white gaussian noise with zero mean.

To select the minimal appropriate lag order , Akaike information criteria is used following (8), where is the number of parameters in the AR model, is the effective number of observations, is the maximum likelihood of the estimate of the error covariance [30, 31]. The best fitted model has the minimum AIC value.


Iv Estimation of the Autoregressive Model Parameters using CF-PSO

Firstly, the structure of the AR model is selected based on (7) for a predetermined order . After that the model parameters are estimated using CF-PSO. During the optimization loop, the algorithm determines the optimal parameters by minimizing the Residual Sum of Squares (RSS) of the estimated model as shown in (9), where is the actual data that need to be predicted and is the estimated data.


The steps of CF-PSO based AR Parameter Estimation are discussed below.

  1. Initialize particle’s position  and velocity in an n-dimensional search space. Here, the dimension n represents the order of the AR model and each vector of the particle position indicates a potential solution.

  2. Calculate the RSS for each ‘initial’ particle positions and determine the particle’s best known position and swarm’s best known position .

  3. Update particle’s position and velocity following (1) and (2) if necessary.

  4. Calculate the RSS with the updated velocity and position .

  5. Repeat (3) and (4) until the stopping criteria is satisfied, i.e., there is not significant change in RSS over a good number of iterations or the algorithm reaches it’s maximum iteration limit. Here maximum iteration is considered 100.

Fig. 1: Optimal LAG selection

V Results and Discussion

The proposed CF-PSO based AR model parameter estimation algorithm is implemented on the practical wind power output data of the Capital Wind Farm, obtained from the Australian Energy Market Operator [32]. The algorithm is implemented using Matlab and standard LS, FB, GL and YW based approaches are evaluated using ‘System Identification Toolbox’ [33].

To compare the performance of the proposed method and four aforementioned well-established methods, following evaluation indices are used:

  1. Mean Square Error (MSE):


    where is the actual data and is the data from the estimated model.

  2. Akaike’s Final Prediction Error (FPE) [13]:


    where, is the number of parameters in the estimated model, is the number of values of the model and

    is the loss function.

  3. Normalized Mean Square Error (NMSE) [34]:


    where is the actual data and is the data from the estimated model. The value of NMSE varies between ‘-inf’ to ‘1’. While ‘-inf’ indicates a bad fit, 1 represents the perfect fit of the data.

Firstly, the appropriate lag order is determined. In order to do that the value of is varied from to and for each value of , the AIC is calculated following the information criterion in (8). Once all AIC value is known, is selected for that fitted AR() model which leads to the minimum AIC value. Considering , in this analysis AIC value is observed when for all four standard methods as shown in Fig. 1. Now, the wind power output data (first week of March 2012 with 5 minute interval) of the ‘capital Wind Farm’ is used to evaluate the performance of the five approaches including the CF-PSO based AR model. For the proposed method, the results are documented considering the mean value of 30 individual runs. The obtained results are summarized in the Table. I and the best results among all approaches are highlighted. Results presented in Table. I shows that the proposed method outperforms the other approaches for this test case. Considering the error value (MSE and FPE) of the LS approach as a base scenario, the Error Minimization Performance (EMP) of other approaches are evaluated following (13), where is the MSE or FPE of the LS method and is the corresponding MSE or FPE of other approaches. The positive value of EMP indicates an improvement of error minimization performance over LS approach while negative value represents that the performance is worse than the LS approach.


In order to justify the performance, the proposed method is employed considering another data set, second week 5s interval data of the month March 2012 for capital wind farm. Results from this method is shown in Table II. In this test case, the proposed CF-PSO based AR model reduces the error indices most. Compared with LS, almost 40% reduction of error is achieved for this test data. Moreover, the NMSE for the standard LS, FB, GL and YW based approaches are around 96.7% while proposed method experiences 98.9%, as shown in Table. II. As the value of NMSE close to unity indicates the best performance, proposed method also shows it superiority for this test case.

(%) (%)
LS 46.7133 46.806 0.975
FB 46.7153 -0.004 46.808 -0.004 0.9751

46.7441 -0.066 46.837 -0.066 0.9751
GL 46.7153 -0.004 46.808 -0.004 0.9751
CF-PSO 45.612 +2.357 45.703 +2.357 0.9757
TABLE I: Performance of AR Parameter Estimation Considering The First Week data of March 2012
(%) (%)
LS 17.17 17.206 0.9669
FB 17.18 -0.0623 17.217 -0.062 0.9669

17.14 0.182 17.175 0.182 0.9669
GL 17.18 -0.067 17.218 -0.067 0.9669
CF-PSO 10.22 +40.487 10.240 +40.487 0.9889

TABLE II: Performance of AR Parameter Estimation Considering The Second Week data of March 2012

Fig. 2 shows the actual wind output data and the estimated model data using the proposed method. The convergence characteristics of the CF-PSO based proposed algorithm is shown in Fig. 3. From the figure, the algorithm convergence within 40 iterations. According to the results shown in Table. I and Table. II, the best improvement is observed considering the CF-PSO based AR modelling among these five approaches for both of the test data sets. The CF-PSO based AR parameter estimation method finds a better solution compared to the gradient based methods due to its global search capabilities. It is important to mention that these well-established gradient based methods may trap in local minima as referred by Huang et al [13]. On the other hand, CF-PSO finds a global optimal solution. In our analysis, we found that the performance of the CF-PSO varied based on the wind data characteristics. If wind data has a global minima that is very close to the local minima, the performance of the CF-PSO is slightly improved compared with other algorithms (as observed in Table. I). On the other hand, if the local minimum is far from the global minima value, significant improvement is observed using the CF-PSO algorithm (as observed in Table. II).

Fig. 2: Comparision of actual data with the estimated model data
Fig. 3: The convergence characteristics of the CF-PSO based proposed algorithm

Vi Conclusion

In this paper, constriction factor based PSO is employed to enhance the performance of the time-series autoregressive model. The proposed algorithm is implemented to estimate the wind power output considering practical wind data. Using the global search capable CF-PSO based proposed AR model, the results obtained in this experiment show that the algorithm finds the solution very accurately and efficiently (within 40 iterations). To justify the results obtained from the proposed method, four algorithms including the widely used Least-Square method and Yule-Walker method are employed for comparison. Experimental results conducted in this experiments show that the proposed method enhances the AR estimation model with better accuracy compared to other four well-established method. In this experiment, the exogenous input variables are not considered during the model estimation, which will be included in the future work. Since the proposed model enhances the performance of the autoregressive model by minimizing model estimation errors more effectively, in the future work the forecasting performance will also be explored in detail.


  • [1] A. Anwar and H. Pota, “Optimum allocation and sizing of DG unit for efficiency enhancement of distribution system,” in IEEE International Power Engineering and Optimization Conference (PEOCO), Jun. 2012.
  • [2] P. Eriksen, T. Ackermann, H. Abildgaard, P. Smith, W. Winter, and J. Rodriguez Garcia, “System operation with high wind penetration,” IEEE Power and Energy Magazine, vol. 3, no. 6, pp. 65–74, 2005.
  • [3] A. Anwar and H. Pota, “Loss reduction of power distribution network using optimum size and location of distributed generation,” in 21st Australasian Universities Power Engineering Conference (AUPEC), Sept. 2011, pp. 1–6.
  • [4] C. Lei and L. Ran, “Short-term wind speed forecasting model for wind farm based on wavelet decomposition,” in Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, 2008.
  • [5] M. Lei, L. Shiyan, J. Chuanwen, L. Hongling, and Z. Yan, “A review on the forecasting of wind speed and generated power,” Renewable and Sustainable Energy Reviews, vol. 13, no. 4, pp. 915 – 920, 2009.
  • [6] D. Hill, D. McMillan, K. R. W. Bell, and D. Infield, “Application of auto-regressive models to u.k. wind speed data for power system impact studies,” IEEE Transactions on Sustainable Energy, vol. 3, no. 1, pp. 134–141, 2012.
  • [7] E. Erdem and J. Shi, “ARMA based approaches for forecasting the tuple of wind speed and direction,” Applied Energy, vol. 88, no. 4, pp. 1405 – 1414, 2011.
  • [8] P. Poggi, M. Muselli, G. Notton, C. Cristofari, and A. Louche, “Forecasting and simulating wind speed in corsica by using an autoregressive model,” Energy Conversion and Management, vol. 44, no. 20, pp. 3177 – 3196, 2003.
  • [9] A. Daniel and A. Chen, “Stochastic simulation and forecasting of hourly average wind speed sequences in jamaica,” Solar Energy, vol. 46, no. 1, 1991.
  • [10] G. Venayagamoorthy, K. Rohrig, and I. Erlich, “One step ahead: Short-term wind power forecasting and intelligent predictive control based on data analytics,” IEEE Power and Energy Magazine, vol. 10, no. 5, pp. 70–78, 2012.
  • [11] T. Senjyu, H. Takara, K. Uezato, and T. Funabashi, “One-hour-ahead load forecasting using neural network,” IEEE Transactions on Power Systems, vol. 17, no. 1, pp. 113–118, 2002.
  • [12] K. Methaprayoon, C. Yingvivatanapong, W.-J. Lee, and J. Liao, “An integration of ANN wind power estimation into unit commitment considering the forecasting uncertainty,” IEEE Transactions on Industry Applications, vol. 43, no. 6, pp. 1441–1448, 2007.
  • [13] C.-M. Huang, C.-J. Huang, and M.-L. Wang, “A particle swarm optimization to identifying the ARMAX model for short-term load forecasting,” IEEE Transactions on Power Systems, vol. 20, no. 2, pp. 1126–1133, 2005.
  • [14] S. Marple, Jr., Digital Spectral Analysis with Applications.   Prentice Hall, Englewood Cliffs, 1987.
  • [15] M. Karen, V. Ramos, and T. Razafindralambo, “Using efficiently autoregressive estimation in wireless sensor networks,” in International Conference on Computer, Information, and Telecommunication Systems (CITS), 2013.
  • [16] G. Box, G. Jenkins, and G. Reinsel, Time Series Analysis: Forecasting and Control

    , ser. Wiley Series in Probability and Statistics.   Wiley, 2008.

  • [17] A. Anwar and H. Pota, “Optimum capacity allocation of DG units based on unbalanced three-phase optimal power flow,” in IEEE PES General Meeting, Jul. 2012.
  • [18] Y. del Valle, G. Venayagamoorthy, S. Mohagheghi, J.-C. Hernandez, and R. Harley, “Particle swarm optimization: Basic concepts, variants and applications in power systems,”

    IEEE Transactions on Evolutionary Computation

    , vol. 12, no. 2, pp. 171–195, 2008.
  • [19] K. Deb, Optimization for engineering design: algorithms and examples.   Prentice-Hall of India, 2004.
  • [20] R. Eberhart and Y. Shi, “Comparing inertia weights and constriction factors in particle swarm optimization,” in Proceedings of the 2000 Congress on Evolutionary Computation, vol. 1, 2000, pp. 84–88.
  • [21] H. Zeineldin, E. El-Saadany, M. Salama, A. Alaboudy, and W. Woon, “Optimal sizing of thyristor-controlled impedance for smart grids with multiple configurations,” IEEE Transactions on Smart Grid, vol. 2, no. 3, pp. 528–537, Sept. 2011.
  • [22] A. Etemadi and M. Fotuhi-Firuzabad, “Distribution system reliability enhancement using optimal capacitor placement,” IET Generation, Transmission Distribution, vol. 2, no. 5, pp. 621–631, 2008.
  • [23] A. Engelbrecht, Computational intelligence: an introduction.   John Wiley & Sons, 2007.
  • [24] M. Clerc and J. Kennedy, “The particle swarm - explosion, stability, and convergence in a multidimensional complex space,” IEEE Transactions on Evolutionary Computation, vol. 6, no. 1, pp. 58–73, Feb 2002.
  • [25] J. Vlachogiannis and K. Lee, “A comparative study on particle swarm optimization for optimal steady-state performance of power systems,” IEEE Transactions on Power Systems, vol. 21, no. 4, pp. 1718–1728, Nov. 2006.
  • [26]

    R. A. Krohling and L. dos Santos Coelho, “Coevolutionary particle swarm optimization using gaussian distribution for solving constrained optimization problems,”

    IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 36, no. 6, pp. 1407–1416, dec. 2006.
  • [27] J. Heo, K. Lee, and R. Garduno-Ramirez, “Multiobjective control of power plants using particle swarm optimization techniques,” IEEE Transactions on Energy Conversion, vol. 21, no. 2, pp. 552–561, june 2006.
  • [28] S. Naka, T. Genji, T. Yura, and Y. Fukuyama, “A hybrid particle swarm optimization for distribution state estimation,” IEEE Transactions on Power Systems, vol. 18, no. 1, pp. 60–68, Feb 2003.
  • [29] S. Xu and Y. Rahmat-Samii, “Boundary conditions in particle swarm optimization revisited,” IEEE Transactions on Antennas and Propagation, vol. 55, no. 3, pp. 760–765, march 2007.
  • [30] P. Chen, T. Pedersen, B. Bak-Jensen, and Z. Chen, “Arima-based time series model of stochastic wind power generation,” IEEE Transactions on Power Systems, vol. 25, no. 2, pp. 667–676, 2010.
  • [31] W. Wei, Time series analysis: univariate and multivariate methods.   Pearson Addison Wesley, 2006.
  • [32] ‘Wind farm performance data.’ [Online]. Available:
  • [33] Matlab, ‘system identification toolbox reference manual.’ [Online]. Available: www.mathworks.com.au
  • [34]

    F. E. Tay and L. Cao, “Modified support vector machines in financial time series forecasting,”

    Neurocomputing, vol. 48, no. 1–4, pp. 847 – 861, 2002.