Dynamic correlations at different time-scales with Empirical Mode Decomposition

by   Noemi Nava, et al.

The Empirical Mode Decomposition (EMD) provides a tool to characterize time series in terms of its implicit components oscillating at different time-scales. We apply this decomposition to intraday time series of the following three financial indices: the S&P 500 (USA), the IPC (Mexico) and the VIX (volatility index USA), obtaining time-varying multidimensional cross-correlations at different time-scales. The correlations computed over a rolling window are compared across the three indices, across the components at different time-scales, at different lags and over time. We uncover a rich heterogeneity of interactions which depends on the time-scale and has important led-lag relations which can have practical use for portfolio management, risk estimation and investments.



There are no comments yet.


page 10

page 11

page 14

page 15


Adaptive Complementary Ensemble EMD and Energy-Frequency Spectra of Cryptocurrency Prices

We study the price dynamics of cryptocurrencies using adaptive complemen...

Inspection of methods of empirical mode decomposition

Empirical Mode Decomposition is an adaptive and local tool that extracts...

Estimation of cluster functionals for regularly varying time series: runs estimators

Cluster indices describe extremal behaviour of stationary time series. W...

Clustering volatility regimes for dynamic trading strategies

We develop a new method to find the number of volatility regimes in a no...

Dynamic mode decomposition for multiscale nonlinear physics

We present a data-driven method for separating complex, multiscale syste...

Sparse Dynamic Distribution Decomposition: Efficient Integration of Trajectory and Snapshot Time Series Data

Dynamic Distribution Decomposition (DDD) was introduced in Taylor-King e...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Correlation structures in financial time series

Financial time series are correlated and the structure of these correlations reflects important market properties mantegna1999introduction , aste2010correlation , aste2006dynamical , aste2010introduction . Financial markets operate at different time horizons di2007multi , and characterizing the relation between market prices at different time-scales is essential to capture the complexity of market dynamics for portfolio management, risk management and investments buonocore2017asymptotic , buonocore2016two , musmeci2014risk , musmeci2016interplay . It is well established and documented that correlations between stock returns varie over time (see for instance aste2010correlation , Longin19953 , Rua2009632 , Vacha2012241 ). It is instead less understood and established how correlations between financial assets vary over time-scales tumminello2007correlation . Most studies only focus on a specific time-scale. However, changes of correlation at different time-scales have important practical consequences. Indeed, if the correlation between two assets varies across time-scales, then market participants with short and long term-horizons have different risk exposures and must adapt their strategies according to the different parts of the correlation spectrum. Furthermore, investigating both the time dependent and the time-scale dependent dynamics of correlations can provide insights on the collective behaviour of traders with varying strategies Longin19953 , bartolozzi2007multi . This is the topic of the present paper where we use a simple methodology to perform this research.

Our approach is similar to what was recently introduced by Chen et al. Chen2010233 who proposed to use the Empirical Mode Decomposition (EMD) to estimate the so-called Time-Dependent Intrinsic Correlation (TDIC). In this approach, two time series are first decomposed into a set of components called Intrinsic Mode Functions (IMFs) oscillating at different time scales. Then, the Pearson correlation is calculated in an adaptive window whose length depends on the instantaneous period of the IMFs. In this paper, we introduce a simplified version of this approach applied to intra-day data (30 seconds) of three indices: the S&P 500 (USA), the IPC (Mexico) and the VIX (volatility index USA). We compute cross-correlations and lagged cross-correlations from different IMFs generated from rolling windows. This yields to dynamic cross-correlations across time scales. The results uncover the presence of cross-scale coupling between the time series and identify some relevant led-lag relation at specific time-scales which could be relevant for practical purposes in portfolio management.

Another technique to measure time-varying correlation and which provides similar outputs is the wavelet coherence Rua2009632 , Vacha2012241 . However, differently from the wavelet transform, the EMD does not require any a priori filter function Peng2005 . The EMD relies on less assumptions, it is a fully data-driven decomposition which can be applied to non-stationary and non-linear data Huang . A similar framework was proposed in horvatic2011detrended where correlations between time series with periodic trends were estimated by using local piecewise polynomial detrending.

This paper is organized as follows. In Section 2, we introduce the basic concepts of the EMD and the IMF. In Section 3, the computation of cross-correlations across time-scales, time-lags and time-windows is described. Section 4 reports the application to real data on three indices: the S&P 500 (USA), the IPC (Mexico) and the VIX. The discussions and conclusions are provided in Section 5.

2 Empirical mode decomposition (EMD)

The EMD method identifies a finite set of oscillations with scale defined by the local maxima and the local minima of the data itself. Each oscillation is empirically derived from the data and is referred to as an Intrinsic Mode Function (IMF). An IMF must satisfy two criteria Huang :

  1. The number of extrema and the number of zero crossings must either be equal or differ at most by one.

  2. At any point, the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero.

The first condition forces an IMF to be a narrow-band signal with no riding waves. The second condition ensures that the instantaneous time-scale will not have fluctuations arising from an asymmetric wave form Huang . The IMFs are obtained through a process called sifting process which uses local extrema to separate oscillations starting with the highest time-scale. Given a time series , , the process decomposes it into a finite number of components, called Intrinsic Mode Functions, here denoted as , , and a residue . The residue is the non-oscillating drift of the data. If the decomposed data consists of uniform scales in the time-scale space, the EMD acts as a dyadic filter and the total number of IMFs is approximately equal to n Flandrin . At the end of the decomposition process, the original time series can be reconstructed as:


The EMD is implemented through the following steps Huang :

  1. Initialize the residue to the original time series and set the IMF index .

  2. Extract the IMF:

    1. initialize and set the iteration counter ;

    2. find the local maxima and the local minima of (see Figure 0(a));

    3. create the upper envelope

      by interpolating between the local maxima and, analogously, create lower envelope

      by interpolating the local minima (see Figure 0(b));

    4. calculate the mean of both envelopes as (see Figure 0(c));

    5. subtract the mean envelope from the input time series, obtaining , see Figure 0(d);

    6. verify if satisfies the IMF’s conditions:

      • if does not satisfy the IMF’s conditions, increase and repeat the sifting process from step (b) (see Figure 0(d));

      • if satisfies the IMF’s conditions, set and define (see Figure 2).

  3. When the residue is either a constant, a monotonic slope or contains only one extrema stop the process, otherwise continue the decomposition from step , setting .

(a) Local maxima and minima.
(b) Upper and lower envelopes.
(c) Envelope mean.
(d) Time series after one sifting step.
Figure 1: Example of one sifting step in the construction of a IMF. (a) Input time series highlighting the local maxima and the local minima. (b) Time series with the interpolated upper and lower envelopes (slashed lines). (c) Time series with the envelopes and the mean of both envelopes (red line). (d) First iteration of the sifting process. In this example, the extracted function does not satisfy the IMF’s conditions and therefore another set of sifting processes must be applied (see Fig.2).

In Figure 1, we exemplify some steps of the sifting process. After one iteration of the sifting process, the function is obtained (Figure 0(d)). In this example, the resulting function is not symmetric and does not have zero mean, hence it is not an IMF yet. Consequently, more iterations of the sifting processes need to be applied to extract the first IMF of the input time series. These further iterations are shown in Figures 1(a), 1(b) and 1(c) with the last sifting iteration which extracts the first IMF, shown in Figure 1(d).

(a) Local maxima and minima.
(b) Upper and lower envelopes.
(c) Envelope mean.
(d) IMF example.
Figure 2: Example of the few final sifting steps which produce a valid IMF. (a) Input time series highlighting the local maxima and the local minima. (b) Input time series with the interpolated upper and lower envelopes (slash lines). (c) Input time series with the envelopes and the mean of both envelopes (red line). (d) Last iteration of the sifting process, the extracted function is the first IMF.

It must be noted that the EMD is based on the timescale separation and does not impose orthogonality, implying that in general the sum of the variance of the components and the residue differs from the variance of the input time series. However, in most practical cases, the difference is small

Huang .

The sifting process eliminates the riding waves and smooths uneven amplitudes Huang . This process terminates when the local mean of the extracted IMF is zero. The difficulty is that this condition can only be approximated and in order to avoid over-sifting and converting meaningful IMFs into meaningless fluctuations with constant amplitude, a stopping criterion needs to be implemented.

3 Cross-correlations on IMF

Let us consider two time series and with , with equal length and with equal intervals of time between observations.

3.1 Cross-correlations across time-scales

The proposed time-scale-dependent correlation computes the Pearson correlation coefficients between two components , , obtained from the decomposition of the time series and , respectively:


where denotes the sample mean over time of and

denotes the sample standard deviation of


Although the IMFs are not theoretically stationary, the IMFs satisfy the condition of having local mean equal to zero and can then be considered to be at least locally stationary Huang . Contrary, the residue does not need to satisfy the IMF conditions, and particularly, for an initial non-stationary time series, the extracted residue will contain the trend of the time series, making it a non-stationary component. Thus, a correlation coefficient between residues is just a measure of linear dependency of the trends indicating if they move in the same direction. This correlation coefficient is likely to be high, and could give misleading results for the interpretation of the dependence structure.

3.2 Time-dependent lagged cross-correlation at the same time-scale over a rolling window

We also compute lagged cross-correlations over a rolling window, which for simplicity, we limit to the same time scale. The cross-correlations between two different time series and , lagged by , over rolling windows of size and at the same time-scale component is defined as:


The time-lag is measured in units of the sampling time-scale. The window approach has the advantage of only assuming local stationarity rather than stationarity over the entire time series. Although this method is based on a simple measure of correlation (Pearson correlation), it adapts to the nature of the data and provides a dynamic measure of correlation across time-scales.

4 Correlation analysis of intraday financial data

We consider intraday data sampled at 30-seconds intervals for two stock market indices and a volatility index, namely, the S&P 500 index (USA), the IPC index (Mexico) and the VIX index (implied volatility index, calculated by the Chicago Board Options Exchange, USA). The data was obtained from Bloomberg BloombergWeb . The observation period includes 184 days, ranging from September 2013 to July 2014 and it only considers the trading days available for all the three indices. Each day has 780 data points (6.5 hours).

Figure 3: Intraday observations (sampled at 30-seconds intervals) for the S&P 500, the IPC and the VIX indices for the time period September 2013 to July 2014.

Figure 3 reports the dynamics of these three indices over that time period. We can observe that the S&P 500 and the IPC indices have similar behaviours. They are indeed positively correlated with correlation coefficient between log-returns equal to 0.21; this is in agreement with previous studies Araujo , Mohamed . On the contrary, the risk-price relationship between the S&P 500 and the VIX indices shows negative correlation, as reported for example in Whaley . The correlation coefficient between these log-returns is equal to -0.26. Finally, the IPC and the VIX indices are essentially uncorrelated with very small negative correlations (the correlation coefficient between log-returns equals to -0.02).

4.1 Intraday analysis of correlation, example for the day July 2014

Let us exemplify the intraday analysis of correlation on a randomly chosen day: July 2014. Figure 4 displays the logarithm of prices for the three indices. Applying the EMD to each time series, we obtained five IMFs and a residue which are reported in Figure 5. The oscillating period of each IMFs is calculated by dividing the total number of points by the number of peaks, with rounded values reported in Table 1.

Figure 4: Intraday log-prices for the S&P 500, the IPC and the VIX indices, example for July 2014.
(a) S&P 500 index.
(b) IPC index.
(c) VIX index.
Figure 5: IMFs of the stock market indices and the volatility index, example for July 2014. From top to bottom IMF1…IMF4 and residue. X-axes time from 9:30 to 16:00.
Index Residue
S&P 4 8 20 44 88
IPC 4 8 16 40 88
VIX 4 8 20 40 88
Table 1: Oscillating period for the IMFs shown in Figure 5 and calculated by dividing the total number of points by the number of peaks, example for July 2014.

4.1.1 Time-scale-dependent correlation, example for July 2014

We computed the time-scale-dependent correlation by means of Equation (2). The results are represented as a matrix of pairwise correlations between the IMFs where the magnitude of the correlation is visually represented by a color-map. Figure 5(a) shows the correlation matrix between the S&P 500 and the IPC indices. We observe positive correlations with mostly larger values on the diagonal (same time-scale components indices) with an increasing magnitude for increasing IMF time-scale. The correlation between the S&P and the VIX indices reveals instead negative values. For these time series, we also observe higher correlations in the diagonal elements which are increasing with increasing time-scale (see Figure 5(b)).

(a) S&P 500 index versus IPC index.
(b) S&P 500 index versus VIX index.
Figure 6: Time-scale-dependent correlation structures, example for July 2014.

4.1.2 Time-dependent correlation, example for July 2014

We also estimated the time-dependent lagged correlation between the S&P 500 and the IPC indices by using Equation (3). These correlations are represented in Figure 6(a) as a color-map matrix in which each column represents a successive window and each row represents a specific time-lag. The intraday correlation values are reported after observations, with the size of the rolling-window. In this way, the size of the correlation matrix is reduced according to the applied window.

Lags are limited to , with and denoting the oscillating period of and , respectively. Choosing larger than the oscillating period results in repetitive patterns in the correlation structure. On the other hand, a shorter time-lag may not reveal some correlations. The window size is set at .

From Figure 6(a), it is difficult to identify correlations patterns for the highest time-scale IMFs. However, for IMFs with lower time-scale, , we observe intervals of stronger correlations characterized by the nature of the oscillating IMFs, i.e., we observe lapses of positive correlation lagged in time by negative values of correlation, making the lead-lag relation between the IMFs almost symmetric with respect to the zero lag.

Figure 6(b) shows the correlation matrices for the S&P 500 and the VIX indices. Contrary to the correlation between the S&P 500 and the IPC indices, the correlation between the S&P and the VIX indices is negative at all frequencies and during the entire trading day. At the highest time-scale, , we observe a clear pattern of negative correlation at lag (1 min), indicating that the S&P 500 leads the VIX index by 1 minute. When correlating the residue components, we observe a dominant blue band, indicating a negative correlation region (a similar red band is observed for the correlation between the S&P 500 and the IPC indices). Such a band could be attributed to the linear and non-stationary characteristics of the residues.

(a) S&P 500 index versus IPC index.
(b) S&P 500 index versus VIX index.
Figure 7: Intraday time-dependent correlation, example for July 2014.

4.2 Intraday correlation, analysis on the complete data set

Proceeding in the same way as in the previous example on July 2014, we decomposed each daily time series into five IMFs and a residue. We computed the time-scale-dependent correlation and the time-dependent correlation for each of the 184 days available in the data set.

4.2.1 Time-scale-dependent correlation

The statistics for the time-scale dependent correlation between the IMFs with the same time-scale index for all trading days are reported in histograms. In Figure 8, we report histograms for the S&P 500 and the IPC indices and in Figure 9 for the S&P and the VIX indices. We observe prevalently positive correlations for the S&P 500 and the IPC components and instead prevalently negative correlations for the S&P 500 and the VIX. The histograms reveal significant deviations from zero for all the components with larger positive or negative correlations for components with longer time-scales.

Figure 8: Distribution of the time-scale-dependent correlation between the IMFs of the S&P 500 index and the IMFs of the IPC index.
Figure 9: Distribution of the time-scale-dependent correlation between the IMFs of the S&P 500 index and the IMFs of the VIX index.

Correlations between the IMF components with different time-scales indices are reported in Figure 10

where the sample median correlations are also reported. We use the sample median of the distribution since this statistic is not influenced by outliers. The case S&P and IPC is shown in Figure

9(a) and the S&P and VIX in Figure 9(b). The color-map matrices at the top are the median correlations between different time-scales whereas the plots below are the values of the diagonal elements (components with same-time scale indices).

(a) S&P 500 index and the IPC index.
(b) S&P 500 index and the VIX index.
Figure 10: Sample median of the time-scale-dependent correlation matrices over the time period from September 2013 to July 2014. The color-map matrices above are the cross correlations between components at different time-scales; the plots below report the values of the elements in the diagonal with same time-scales indices.

4.2.2 Rolling window analysis and lag relations

We analysed the median of the time-dependent correlation matrices, computed as reported in Equation 3, over rolling windows and with lags. The window sizes and the time lags which were used are reported in Table 2.

Component S&P vs IPC S&P vs VIX
Lag Window Lag Window
4 20 4 20
9 20 9 20
19 20 21 21
44 44 48 48
110 110 124 124
Residue 110 110 124 124
Table 2: Average of the number of lags and the size of the rolling-window used for the time-dependent correlation analysis.

The median, time-varying, lagged correlation matrix (Eq.3) between the S&P 500 and the IPC indices is displayed in Figure 10(a). Overall, we observe relatively small correlations with little lead-lag relationships at all time-scales with larger values for the last two components and the residual (bottom panels). We observe patterns in the intraday activity with less persistent correlations around the middle of the day.

More intense negative correlation is observed between the S&P 500 and the VIX indices, reported in Figure 10(b). Interestingly, in this case, we observe significant lagged correlations at small time scales (, and ) with the S&P 500 leading the VIX at one minute lag () with a stable pattern across the day. This indicates that consistently changes in the S&P 500 are followed by changes in the VIX after about 1 min and in the opposite direction (negative correlations).

(a) S&P 500 index versus IPC index.
(b) S&P 500 index versus VIX index.
Figure 11: Sample median of the time-dependent correlation matrices over the time period from September 2013 to July 2014.

5 Discussions and conclusions

In this paper we propose a simple approach which shows that Empirical Mode Decomposition can be used to investigate the correlation between time series at different time-scales. This expands the concept of correlations to a higher-dimensional level. We observed that, although most of the correlation is between components of the same time-scales, there are some significant correlations also between components of different time-scales. A dynamical analysis performed over rolling windows of 30 seconds shows that correlations’ patterns are both time and time-scale dependent. We uncovered lead-lag relations within components with the discovery of a persistent and significant 1 min negative coupling between the S&P 500 and the VIX indices which can have practical relevance for trading strategies and risk modeling.

The methodology we introduced in this paper and our findings are consistent with the coherence measure obtained with the wavelet transform Vacha2012241 . However, given the simplicity of the EMD method and its adaptability to different time series without needing to specify any filter function, we believe that the proposed correlation measures offer a simpler, computationally more efficient and easier to interpret approach.

The measures proposed in this paper and in particular Eqs. 2, 3 are the simplest generalisations of the linear correlation measure to include time-scale components. We chose them as a natural extension of the cross-correlation concept. However, there are some aspects of the present approach that would be interesting to further investigate in future works. For instance, Eqs. 2, 3

perform averages over the variables, but time-series with different oscillation scales lead to different averages even if the (scaled) nature of the variable is the same. This is probably penalising the values associated with high frequency components that we indeed observe to be consistently smaller.


The authors wish to thank Bloomberg for providing the data. NN would like to acknowledge the financial support from Conacyt-Mexico. TDM wishes to thank the COST Action TD1210 for partially supporting this work. TA & TDM wish to thank the Systemic Risk Centre at LSE.


  • [1] Rosario N Mantegna and H Eugene Stanley. Introduction to econophysics: correlations and complexity in finance. Cambridge university press, 1999.
  • [2] Tomaso Aste, W Shaw, and T Di Matteo. Correlation structure and dynamics in volatile markets. New Journal of Physics, 12(8):085009, 2010.
  • [3] Tomaso Aste and T Di Matteo. Dynamical networks from correlations. Physica A: Statistical Mechanics and its Applications, 370(1):156–161, 2006.
  • [4] Tomaso Aste and T Di Matteo. Introduction to complex and econophysics systems: A navigation map. Complex physical, biophysical and econophysical systems, pages 1–35, 2010.
  • [5] T Di Matteo. Multi-scaling in finance. Quantitative finance, 7(1):21–36, 2007.
  • [6] RJ Buonocore, T Aste, and T Di Matteo. Asymptotic scaling properties and estimation of the generalized hurst exponents in financial data. Physical Review E, 95(4):042311, 2017.
  • [7] RJ Buonocore, N Musmeci, T Aste, and T Di Matteo. Two different flavours of complexity in financial data. The European Physical Journal Special Topics, 225(17-18):3105–3113, 2016.
  • [8] Nicoló Musmeci, Tomaso Aste, and T Di Matteo. Risk diversification: a study of persistence with a filtered correlation-network approach. arXiv preprint arXiv:1410.5621, 2014.
  • [9] Nicoló Musmeci, Tomaso Aste, and T Di Matteo. Interplay between past market correlation structure changes and future volatility outbursts. Scientific reports, 6, 2016.
  • [10] François Longin and Bruno Solnik. Is the correlation in international equity returns constant: 1960-1990? Journal of International Money and Finance, 14(1):3–26, 1995.
  • [11] Antonio Rua and Lucas C. Nunes. International comovement of stock market returns: A wavelet analysis. Journal of Empirical Finance, 16(4):632–639, 2009.
  • [12] Lukas Vácha and Jozef Baruník. Co-movement of energy commodities revisited: Evidence from wavelet coherence analysis. Energy Economics, 34(1):241–247, 2012.
  • [13] Michele Tumminello, T Di Matteo, Tomaso Aste, and Rosario N. Mantegna. Correlation based networks of equity returns sampled at different time horizons. The European Physical Journal B, 55(2):209–217, 2007.
  • [14] Marco Bartolozzi, Christopher Mellen, T Di Matteo, and Tomaso Aste. Multi-scale correlations in different futures markets. The European Physical Journal B, 58(2):207–220, 2007.
  • [15] X. Chen, Z. Wu, and N.E. Huang. The time-dependent intrinsic correlation based on the empirical mode decomposition. Advances in Adaptive Data Analysis, 2(2):233–265, 2010. cited By 19.
  • [16] Z.K. Peng, Peter W. Tse, and F.L. Chu. A comparison study of improved Hilbert-Huang transform and wavelet transform: Application to fault diagnosis for rolling bearing. Mechanical Systems and Signal Processing, 19(5):974–988, 2005.
  • [17] Norden E. Huang, Zheng Shen, Steven R. Long, Manli C. Wu, Hsing H. Shih, Quanan Zheng, Nai-Chyuan Yen, Chi Chao Tung, and Henry H. Liu. The empirical mode decomposition and the Hilbert spectrum for non-linear and non-stationary time series analysis. Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 454(1971):903–995, 1998.
  • [18] Davor Horvatic, H Eugene Stanley, and Boris Podobnik. Detrended cross-correlation analysis for non-stationary time series with periodic trends. EPL (Europhysics Letters), 94(1):18007, 2011.
  • [19] P. Flandrin, G. Rilling, and P. Goncalves. Empirical mode decomposition as a filter bank. Signal Processing Letters, IEEE, 11(2):112–114, Feb 2004.
  • [20] https://www.bloomberg.com/professional/product/reference-data/
  • [21] Eurilton Araújo. Macroeconomic shocks and the co-movement of stock returns in Latin America. Emerging Markets Review, 10(4):331–344, 2009.
  • [22] Mohamed El Héidi Arouri, Amine Lahiani, and Duc Khuong Nguyen. Cross-market dynamics and optimal portfolio strategies in Latin American equity markets. European Business Review, 27(2):161–181, 2015.
  • [23] Robert E Whaley. The investor fear gauge. The Journal of Portfolio Management, 26(3):12–17, 2000.