Construction of Minimum Spanning Trees from Financial Returns using Rank Correlation

05/08/2020 ∙ by Tristan Millington, et al. ∙ University of Southampton 0

The construction of minimum spanning trees (MSTs) from correlation matrices is an often used method to study relationships in the financial markets. However most of the work on this topic tends to use the Pearson correlation coefficient, which relies on the assumption of normality and can be brittle to the presence of outliers, neither of which is ideal for the study of financial returns. In this paper we study the inference of MSTs from daily US financial returns using Pearson and two rank correlation methods, Spearman and Kendall's τ. We find that the trees constructed using these rank methods tend to be more stable and maintain more edges over the dataset than those constructed using Pearson correlation, that there are significant differences in the agreement of the centrality of various sectors and that despite these, the trees tend to have similar topologies.



There are no comments yet.


page 15

page 16

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Investors tend to not invest purely in single asset, but due to a desire to reduce risk and increase diversification, own portfolios made up of multiple assets. To accurately assess risk in these portfolios we must understand the dynamics of the relationships between said assets. Various methods of inferring the strength of relationships exist, for instance correlation Mantegna (1999) Onnela et al. (2004), partial correlation Millington and Niranjan (2019) Kenett et al. (2010) Millington and Niranjan (2020) or mutual information Fiedor (2014) Guo et al. (2018), but Pearson correlation is the most ubiquitous. From these asset to asset relationships we can use network theory to study the system as a whole.

Accurate inference of these correlation matrices from a dataset with assets and samples is challenging for numerous reasons. If is not significantly larger than , the correlation matrix will contain significant amounts of noise. Due to non-stationarities in the markets, we often want a small window of data, where we can assume that the data is stationary Tsay (2005). This often results in being close to , even if we have a large amount of data. Options to help remove noise in this situation include topological filtration methods (e.g. Minimum Spanning Trees Mantegna (1999), Planar Maximally Filtered Graphs Tumminello et al. (2010)

), random matrix theory approaches

Namaki et al. (2011) Laloux et al. (2000) Plerou et al. (2002) or thresholding Boginski et al. (2005) Onnela et al. (2004). In this paper we focus on the use of topological methods for this purpose, due to the simplicity of construction and interpretation. To start with, we follow Mantegna Mantegna (1999) and construct a distance matrix () from the correlation matrix ():


This distance matrix is then used as the adjacency matrix for a new graph, the distance graph. From this we can use Kruskal’s algorithm to create a minimum spanning tree (MST), which proceeds as follows

  • Initialise the tree as a disconnected graph made up of the nodes in the distance graph

  • Sort edges of the distance graph in ascending order and place in a list

  • For each edge between and in this sorted list

    • If and are not in the same component in the tree, add this edge into the tree

Various ’stylised facts’ are known about these trees, for instance

  • Branches of the trees tend to contain companies in the same sector Mantegna (1999)

  • The trees shrink and have different structures during times of market stress compared to market calm Onnela et al. (2003a) Zhang et al. (2011)

  • The trees tend to have a ’scale free’ structure, with nodes of high degree (hubs) occurring more than would be expected from a random graph Bonanno et al. (2003) Vandewalle et al. (2001) Onnela et al. (2003a)

  • Assets with large weights in Markowitz portfolios tend to be peripheral Onnela et al. (2003b) Hüttner et al. (2018) Peralta and Zareei (2016), however there is disagreement over whether to chose assets on the peripheries or center of the networks for better Sharpe ratios Kaya (2013) Pozzi et al. (2013) Peralta and Zareei (2016)

  • The MSTs tend to only keep significant correlations Aste et al. (2010)

While most of the focus has been on the US markets, MST based models have been applied to markets from other countries (e.g. Japan Jung et al. (2008), the UK Coelho et al. (2007), Italy Coletti (2016) and South Korea Jung et al. (2006)), to cryptocurrencies Stosic et al. (2018) Song et al. (2019) and to networks from neuroscience Tewarie et al. (2014) Tewarie et al. (2015).

While the interpretability of the Pearson correlation coefficient is a big plus, it assumes normality, something which most assets return distributions do not follow Cont (2001), and is sensitive to outliers. There are of course correlation measures that do not suffer from these issues, namely those based on rank. Rank correlation methods calculate the correlation between the ranks of variables, which tends to remove the effects of outliers while still giving a measure of the degree to which two variables increase or decrease together. However to the best of our knowledge, most of the literature which studies the correlations between asset returns tends to use the Pearson correlation coefficient. Therefore in this paper we compare networks inferred from stock returns using Pearson, Spearman and Kendall’s correlation in order to see if the robustness of these rank correlations can improve our understanding of the stock markets. Previous work Miccichè et al. (2003) has briefly mentioned that MSTs constructed using Spearman correlation from volatility measures of stocks are more robust, but they did not explicitly compare the two correlation coefficients. In a paper more broadly looking at the effects of weighting observations, Pozzi et al. Pozzi et al. (2012) compare Pearson correlation and Kendall’s . They find that matrices constructed using Kendall’s tend to contain more information than those constructed using Pearson correlation, and are affected less when they weight observations.

A paper on a more similar theme to this is written by Musmeci et al. Musmeci et al. (2017), who take a multilayer network approach. Each layer is composed of a Planar Maximally Filtered Graph, constructed using a different method. Four methods are used to quantify relationships between assets, Pearson correlation, Kendall’s , tail dependence and partial correlation. They find that these layers tend to have significant differences, with between 30% and 70% of the edges being unique to each layer. Pearson and Kendall’s tend to be the methods that agree the most, with a correlation of around 0.7 on the degree of nodes. Interestingly they find that the level of agreement drops during times of crisis, showing that these different methods tend to pick up different signals from the markets and indicating that being mindful of multiple methods of quantifying relationships is valuable when taking a network approach to financial returns.

The final example we found is by Shirokikh et al. Shirokikh et al. (2013) who use a thresholding model with Spearman correlation, but they do not compare how this model differs to a Pearson based one.

The Pearson correlation between two variables and is defined as follows


To calculate the Spearman correlation, we firstly sort the values, replace each value with its rank, and calculate the Pearson correlation between the ranks. This then measures the degree to which two variables monotonically increase or decrease together. Kendall’s is slightly more complicated, measuring the relationship by considering the number of concordant pairs vs the number of discordant pairs. A pair of observations is concordant if and or if and . It is discordant if and or if and . The -a formula simply counts the number of concordant pairs vs the number of discordant pairs, divided by the total number of pairs. This does not however take into account any ties that might occur in the data, so we use the -b formulation, defined as


where is the number of concordant pairs, is the number of discordant pairs, , , , is the number of values in the th group of ties for variable , is the number of values in the th group of ties for variable . Any reference to in the rest of the paper refers to this -b formation.

2 Data and Software

The data we use is downloaded from Yahoo Finance. We use log returns from the S&P500 from 2000/03/01 to 2019/10/21. Any company missing more than 10% of its data is removed, and any missing values are filled forwards from the first good value. If the values are missing from the start we backfill from the first good value. This results in 4790 days of return data for 229 companies. From this data we take a window of 504 days and slide along 30 days at a time, creating 142 windows to infer correlations from. Each company is tagged with a sector from the GICS classification using information from Bloomberg. This places each company into 1 of 11 sectors, Information Technology, Real Estate, Materials, Communication Services, Energy, Financials, Utilities, Industrials, Consumer Discretionary, Healthcare or Consumer Staples.

We make use of Python, NumPy and SciPy Oliphant (2006) for general scripting, pandas McKinney (2010) for handing the data, statsmodels Seabold and Perktold (2010) for some of the statistical analysis, matplotlib Hunter (2007) for plotting, arch Sheppard et al. (2019) for the implementation of the circular bootstrap, Networkx Hagberg et al. (2008) for the network analysis and gephi Bastian et al. (2009) for the MST visualization. The code and data is available at

3 Results and Analysis

3.1 Correlation Matrix Analysis

Firstly we analyze the full correlation matrices with no filtration. A starting point is to look at the correlation coefficient for the same set of values. Figure 1 shows a set of scatter plots comparing the correlations. From this we can see there is a degree of agreement between all, and generally larger correlations are more likely to be similar. However there is a ’fat’ middle when comparing the rank correlations to the Pearson correlation, where there can be significant disagreement. Spearman and seem to be very similar, with there being a strong relationship between the two.

(a) Pearson vs Spearman
(b) Pearson vs
(c) Spearman vs
Figure 1: Relationship of the correlation coefficients from the entire dataset. There is obviously a degree of agreement, but the ’fat’ middle is notable when comparing the Pearson and rank correlations, with it being possible to have a relatively large and significantly correlation by one coefficient, and a smaller and less significant correlation in the other measure. The rank correlations themselves are relatively similar.

The largest eigenvalue of the correlation matrix is a measure of the intensity of the correlation present in the matrix, and in matrices inferred from financial returns tends to be significantly larger than the second largest

Laloux et al. (2000) Plerou et al. (2002). Generally this largest eigenvalue is larger during times of stress and smaller during times of calm Drożdż et al. (2000) Laloux et al. (2000). Firstly we study how this varies over time for each correlation measure. This is shown in Figure 2. For all of the networks there is a similar shape, with it peaking during times of market stress and dropping during times of calm. The Spearman and Pearson correlations have relatively similar values, although the Spearman has a smaller range. The correlation is much smaller than the other two at all times, and also has a smaller range. Times of stress and volatility tend to bring more outliers in returns data, which could be the cause of the difference in largest eigenvalue, but it is interesting that the eigenvalue for correlation is so much smaller at all times.

Figure 2: Largest eigenvalue () in the networks over time. From this we can see the Spearman correlation has a slightly smaller largest eigenvalue than the Pearson correlation, while the

correlation is much smaller. The rank methods also have a smaller range than the Pearson correlation. The volatility of the markets at times of stress is likely to lead to more outliers, so the robustness of the rank correlations to these could be causing the reduced variance of the largest eigenvalue.

3.2 MST Stability

From these inferred correlation matrices we transform them into distance matrices using (1) and construct minimum spanning trees using Kruskal’s algorithm. In this section we analyze and compare the stability of the trees constructed using the various coefficients. Example MSTs from the first window of data are shown in Figure 3. In all there is clear sector clustering, with branches of the trees consisting of nodes from the same sector. This has been noted before Mantegna (1999).

(a) Pearson MST
(b) Spearman MST
(c) MST
(d) Node colour legend
Figure 3: Example MSTs constructed from the various correlation coefficients. Nodes are coloured according to their sector membership. From this colouring we can see that all coefficients show a strong degree of sector clustering, with branches of the trees tending to contain companies in the same sector, and the presence of several important hub nodes.

Firstly we focus on measuring the fraction of edge changes between MSTs adjacent in time, which quantifies how stable the trees are, and how well change in the market is detected. The results are plotted in Figure 4. Here we can see a large difference, with the MSTs inferred using the rank correlations showing more stability over time than those inferred using Pearson correlation. This is particularly noticeable during 2009, where the markets were very volatile due to the financial crisis. Around 50% of the edges change for the Pearson MSTs, but only 30% change for the Spearman MSTs, and 25% for the MSTs. It is also interesting to note that the edge difference rises at the start of the crisis and then drops during the actual crisis itself for the Pearson and Spearman MSTs, and just drops for the MSTs. Other authors have noted that by some measures the markets could be considered more stable during these times Drożdż et al. (2000) Kocheturov et al. (2014), but we have not found that this has been mentioned in the context of MSTs.

This therefore shows that the Spearman and MSTs tend to be more consistent than Pearson MSTs. This is particularly prominent at the start of the financial crisis, with the Pearson MSTs showing a large spike in difference, while the Spearman and MSTs show little or no change in difference. In this particular situation we would expect the heavy tails to affect the Pearson correlation between two assets more than the rank methods, and this should change the edges selected by the MST construction procedure.

Figure 4: Edge difference between adjacent MSTs. From this we can see the MSTs inferred using rank correlation are far more stable with regards to time than those inferred using Pearson correlation. While all of the trees seem to become more similar during the financial crisis, the Pearson MSTs show a big reconfiguration as the crisis starts, while the MSTs shows no spike before dropping.

Next we measure how the MSTs have changed from the first inferred tree using the fraction of edges that differ from the first tree to the current tree. This measures the life of an edge and shows us how the tree evolves. A plot of this is shown in Figure 5. From this we can see that quite rapidly the trees differ from the original, with 70% of the edges changing within 2 years. For our experiments, what is particularly interesting is that the rank MSTs maintain slightly more edges than the Pearson MSTs, but the difference between the and Spearman MSTs is very small.

Figure 5:

Fraction of edges that differ in the tree inferred at that moment in time from the first tree inferred. This gives us a measure of how long the edges persist for. Most of the edges disappear very rapidly, with around 70% changing within 2 years. The rank MSTs seem to be slightly more stable than the Pearson MSTs, maintaining more edges from the initial tree, but the difference between Spearman and Kendall’s

methods is very small.

Over the entire dataset, the Pearson MSTs maintain 4 edges, the Spearman MSTs maintain 7 edges and the MSTs maintain 8 edges. For the Pearson MSTs these edges are

  • T - VZ (Communication Services - Communication Services)

  • CCL - RCL (Consumer Discretionary - Consumer Discretionary)

  • DHI - LEN (Consumer Discretionary - Consumer Discretionary)

  • HD - LOW (Consumer Discretionary - Consumer Discretionary)

In addition to these, the Spearman MSTs also maintain

  • CAT - DE (Industrials - Industrials)

  • FDX - UPS (Industrials - Industrials)

  • GS - MS (Financials - Financials)

The MSTs maintain all the edges mentioned so far, plus

  • HAL - SLB (Energy - Energy)

It is notable these are all intrasector edges, and that there is a large overlap between all the trees - all of the edges maintained by the Pearson MSTs are maintained by the Spearman MSTs, and all of the edges maintained for the Spearman MSTs are maintained for the MSTs.

There is of course the question of how the difference between the MSTs changes over time. We measure the fraction of edges that differ between the three MSTs and plot it in Figure 6. From this we can see there is a significant difference in the presence of edges between the rank MSTs and the Pearson MSTs. The difference does seem to increase during the financial crisis, with peaks occurring during 2008 and 2009. There seems to be relatively little difference between the two rank MSTs, with less than 10% of the edges being different for most of the dataset. This difference between the rank methods does not seem to be particularly affected by market conditions.

Figure 6: Fraction of the edges that differ between the various MSTs. There is a large degree of disagreement between the Pearson and rank-based correlations, even at the smallest difference around 30% of the edges differ. The difference peaks during the financial crisis, where over 50% of the edges differ at one point between the Pearson and rank MSTs. The difference between the rank MSTs themselves is small, with less than 10% of the edges differing.

With this knowledge that the MSTs select quite different edges, we next ask which edges are selected differently. To start with we focus on the agreement between the degree of the nodes (the number of edges each node has). If we plot the Spearman correlation between the node degrees (Figure 6(a)) we can see that there is relatively high agreement between node degrees, aside from during 2011. This implies that nodes tend to be regarded as similarly important. However this does not show if this is in the core of the network or the peripheries. To measure this we set a threshold to remove high degree nodes (hubs), in this case arbitrarily defined as degree values over 4, and plot the correlation in Figure 6(b). We can see that the correlation drops significantly between the Pearson MSTs and the rank MSTs, indicating that it is the hub nodes that are the greatest source of agreement. If this threshold is reversed, and the low degree nodes removed, the correlation is maintained at around 0.7.

(a) All
(b) Peripheries
Figure 7: Correlation between node degrees from the MSTs. Left shows the correlation between the degree of all nodes, while right shows only the correlation between the peripheries (defined as nodes with a degree of less than 5). Overall (left) we can see there is relatively high agreement on the node degrees, although as above the trees diverge during the financial crisis, while the reduction in correlation (with an average of 0.4) when removing the hubs compared to the overall correlation (with an an average of 0.7) shows most of the disagreement comes at the peripheries

3.3 Node and Sector Centrality

Now we look at the centrality of the nodes and economic sectors in the trees. If we sum the degree of each node for each MST we can get a measure of degree over the entire dataset, showing us which nodes are regarded as important overall. We refer to this as the total degree. The Spearman correlation between the total degrees is high (Pearson/Spearman = 0.94, Pearson/ = 0.94, Spearman/ = 0.99) indicating that the MSTs generally agree on the total centrality of most nodes. The 10 nodes that have the largest total degree shown in Table 1. We can see there is a significant overlap between the trees, with 5/10 shared between all, and 9/10 shared between the rank MSTs.

Company Sector Degree Sum
ITW Industrials 986
PPG Materials 980
PH Industrials 741
HON Industrials 716
ETN Industrials 712
BLK Financials 647
SPG Real Estate 638
TROW Financials 629
JPM Financials 625
UTX Industrials 611
Company Sector Degree Sum
PPG Materials 999
PH Industrials 862
MMM Industrials 721
ITW Industrials 677
ETN Industrials 674
TROW Financials 658
CAT Industrials 642
BEN Financials 604
AVB Real Estate 588
JWN Consumer Discretionary 562
Company Sector Degree Sum
PPG Materials 970
PH Industrials 874
MMM Industrials 726
TROW Financials 700
ITW Industrials 691
ETN Industrials 661
CAT Industrials 621
BEN Financials 583
LNC Financials 579
AVB Real Estate 579
Table 1: Nodes with the highest total degree in the Pearson (top), Spearman (middle) and (bottom) MSTs. Bold rows are shared between all, italic rows are shared between two. There is a strong overlap - 5 out of 10 are shared between all 3 indicating that generally they agree on which nodes are the most important, and 9 out of 10 are shared between the rank MSTs.

On a sector note we can see the Industrials sector seems particularly central in these MSTs, with 5 of the top 10 in all being made up of companies from this sector. The next closest is the Financials sector, making up 3 of the top 10 in the Pearson and MSTs and 2 in the Spearman MSTs. The Materials and Real Estate have 1 entry in the top 10 for all 3 MSTs, and the Consumer Discretionary appears once in the Spearman table.

From this we can look at how the centrality of a sector varies over time. In particular we look at how the mean centrality varies by calculating the centrality of all the nodes in a sector and taking the mean. This reduces the effect of the different numbers of companies in each sector. To measure this we use both degree centrality and betweenness centrality. Betweenness centrality is calculated by looking at the fraction of shortest paths that pass through a node, and allows us to get a different perspective on which edges are regarded as important. The results are shown in Figures 8 (degree centrality) and 9 (betweenness centrality).

Focusing on the mean sector degree centrality (Figure 8) we can see that the Financials sector is important in all 3 MSTs. For the Pearson MSTs it becomes important in 2004, and particularly important in 2008. For the rank methods it becomes very important in both 2004 and 2008, but the peak in 2008 seems smaller. The Industrials sector is usually important for most of the dataset, but it does seem more variable for the Pearson MSTs, with a strong peak in 2012. This is less visible in the rank trees, particularly the Spearman MSTs. The Communication Services sector is generally not central in any of the trees, with brief exceptions for 2017 in the Pearson MSTs and 2011 in the Spearman and MSTs. Another point of disagreement is the Information Technology sector, which does not become significantly central for the Pearson MSTs, but does for the rank trees, particularly during 2013.

Next we look at the mean sector betweenness centrality (Figure 9). Compared to the mean degree centrality there is a far larger spread of values - looking at the legend the values go from 0 - 0.18 for betweenness centrality vs 0 - 0.0018 for the degree centrality. The peaks are also much more noticeable. A particularly noteworthy peak is that of the Energy sector in the Pearson MSTs during 2010, which is not shared in either of the rank methods. All MSTs generally seem to regard the Financials sector as central, but there is a strong peak for the MSTs during 2002 that is less present for either the Pearson or the Spearman MSTs. There are peaks for Real Estate and Consumer Staples in the Pearson MSTs during 2018 that are not present in the rank MSTs. All three pick up a period between 2009 and 2011 where the centrality of the Materials sector increases.

(a) Pearson (b) Spearman (c)
Figure 8: Mean degree centrality for the sectors over time.
(a) Pearson (b) Spearman (c)
Figure 9: Mean degree centrality for the sectors over time.

In general it seems there more agreement between the Spearman and MSTs than between Pearson and either one of them. There is also less agreement on mean sector centrality when using betweenness centrality vs degree centrality. To quantify this further we calculate the Spearman correlation between the mean centralities over time, and show the results in Tables 2 (degree centrality) and 3 (betweenness centrality).

For the degree centrality (Table 2) the Spearman and methods strongly agree on the sector centralities, with a mean Spearman correlation of . This is much weaker for the Pearson and MSTs () and the Pearson and Spearman MSTs (). The agreement for the Communication Services sector is notably low between the Pearson and rank MSTs, but even between the rank MSTs it is the lowest of all the sectors. However it is one of the smallest sectors, so the poor or excellent performance of a single company is less likely to be hidden by other companies in the sector. There is strong agreement in the Health Care, Consumer Staples, Real Estate and Industrials sectors between all of the MSTs.

For the betweenness centrality (Table 3) again there is a generally strong agreement on the sector centralities between the Spearman and MSTs, with the mean Spearman correlation between the sector centralities being . The Pearson - mean correlation is and the Pearson - Spearman mean correlation is , showing a much weaker relationship. Again Communication Services has the lowest agreement between all of the methods, and there is low agreement between the centrality of the Industrials, Energy and Financials sectors for the Pearson and rank MSTs. These also have a large drop in agreement in betweenness centrality compared to degree centrality. It is particularly notable as these sectors are usually regarded as important and central to the US economy, and also as there was relatively strong agreement in the mean degree centralities between the MSTs for the Industrials sector. In general there is a drop in mean correlation when using betweenness centrality as a measure when compared to degree centrality. This implies that companies tend to take different positions in the Pearson MSTs compared to the rank MSTs. There is however a much greater range in the betweenness centralities than the degree centralities, which may affect the correlation.

Sector Pearson - Spearman Pearson - Spearman -
Industrials 0.779 0.801 0.956
Health Care 0.768 0.754 0.973
Financials 0.664 0.655 0.962
Materials 0.391 0.538 0.893
Real Estate 0.715 0.664 0.942
Consumer Staples 0.824 0.810 0.977
Utilities 0.430 0.387 0.919
Information Technology 0.657 0.654 0.946
Energy 0.543 0.551 0.911
Communication Services 0.181 0.190 0.763
Consumer Discretionary 0.545 0.526 0.936
Table 2: Correlation between the mean degree centrality for each sector between the MSTs. There seems to be reasonably broad agreement between all the MSTs on the centrality of most sectors, with the exceptions of Communication Services and Materials. The rank MSTs show a large amount of agreement, while the Pearson MSTs tend to differ more.
Sector Pearson - Spearman Pearson - Spearman -
Industrials 0.254 0.259 0.895
Health Care 0.596 0.594 0.949
Financials 0.396 0.425 0.886
Materials 0.405 0.487 0.815
Real Estate 0.541 0.546 0.862
Consumer Staples 0.785 0.764 0.946
Utilities 0.540 0.563 0.893
Information Technology 0.392 0.395 0.778
Energy 0.316 0.379 0.803
Communication Services 0.109 0.067 0.802
Consumer Discretionary 0.527 0.492 0.889
Table 3: Correlation between the mean betweenness centrality for each sector between the MSTs. There is a significant drop in the mean agreement of the betweenness sector centralities compared to the degree sector centrality, particularly for the Industrials sector. There is still generally high agreement between the rank MSTs, but the Pearson MSTs seem to have a significant difference.

3.4 MST Topology

Having studied the stability of the trees over time and the importance of various sectors, we now look if the structure of the MSTs differ using some network measures. We use the leaf fraction (fraction of nodes with only 1 edge), exponent when fitting a power law to the degree distribution, the average shortest path length and mean occupation layer (the mean of all shortest paths from each node to the center of the tree). In this case we take the center of the tree as the node with the largest degree. The plots of these measures over time are shown in Figure 10.

(a) Average shortest path length
(b) Mean Occupation Layer
(c) Leaf Fraction
(d) Exponent
Figure 10: Topological measures of the structure of the MSTs. From this we can see all the correlation coefficients produce MSTs with very similar structures. Macroeconomic effects are visible in the trees, with the average shortest path length and leaf fraction decreasing during the financial crisis, although the mean occupation layer seems less sensitive.

From these we can see that irrelevant of the coefficient, the MSTs have similar structure. All of the trees have a heavy tailed degree distribution, with there being a high number of nodes with only one other edge and a small number of edges with a large degree. We can see the structures change during the financial crisis, with the leaf fraction and average shortest path length decreasing, although there is no noticeable change for the mean occupation layer.

3.5 MST Robustness

Finally we are interested in comparing the robustness of these correlation coefficients. This can be done using a bootstrap based approach, in a similar manner to Tumminello et al. Tumminello et al. (2007) and Musciotto et al. Musciotto et al. (2018)

. Here we create 1000 pseudo-datasets using a circular bootstrap. A circular bootstrap is a type of block bootstrap where we select data from a continuous stretch of time, and if the end of the dataset is reached we wrap round and start back at the beginning. This tends to be more appropriate for time series data compared to the classic bootstrap due to look ahead effects and potential autocorrelation. With these pseudo-datasets we calculate the correlations between assets and construct MSTs from these correlation matrices. Once we have this set of MSTs, we can compare the edges present in them. Ideally if there is no noise, the data is purely stationary and the methods robust all these MSTs would be the same. Of course this is not the case in real life. To run the bootstrap we take the first 1008 days of data and create 1000 bootstrapped datasets of 504 days. Firstly we measure the mean and standard deviation of the fraction of difference in edge presence across the trees. The results are shown in Table

4. From this we can see that the rank MSTs select slightly more of the same edges, but the difference is not large.

We can gain a measure of how likely an edge is to exist in these MSTs by counting the number of times an edge exists in our forest of trees and diving by the total number of bootstrap replications. This gives us a p-value for the edge. Previous work Tumminello et al. (2007) has shown that there is little relationship between p-value and the edge weight in Pearson correlation based MSTs, so we explore this for the other MSTs. The results are shown in Table 4. Again there seems to be little correlation between a p-value and the edge weight for any coefficient, and all have a similar correlation between p-value and edge weight.

Method Mean Difference Correlation
Pearson 0.417 0.136 0.276
Spearman 0.388 0.124 0.268
0.386 0.121 0.268
Table 4: Mean difference between MSTs constructed from the bootstrap datasets and correlation between p-value and edge weight. The rank MSTs tend to select the same edges slightly more from the bootstrap datasets than the Pearson MSTs, but the difference is not large. None of the correlation coefficients have a particularly strong relationship between p-value and edge weight.

Companies in the same sector tend to be more correlated than those in different sectors and so we might expect those edges to have a larger p-value than the inter sector ones. To measure this we take the mean of the p-values of the edges in the tree inferred from the overall dataset for all edges, intrasector edges and intersector edges. The results are shown in Table 5. From this we can see the rank MSTs have a larger total p-value compared to Pearson MSTs, although the difference between is not very large. The rank MSTs also have a larger mean p-value for intrasector edges compared to the Pearson MSTs, although all of the differences are within the standard deviations.

Method Total Intrasector edges Intersector edges
Pearson 0.682 0.229 0.706 0.222 0.572 0.230
Spearman 0.705 0.232 0.737 0.216 0.574 0.253
0.705 0.238 0.741 0.215 0.556 0.269
Table 5: Mean and standard deviation of p-values found for the edges from the MSTs inferred from the first window of data. For all correlation methods intrasector edges are more likely than intersector ones, but interestingly the rank-based correlation MSTs have a slightly higher p-value for an intrasector edge than the Pearson MSTs.

4 Conclusion

In this paper we have used the Pearson, Spearman and Kendall’s correlation coefficients to infer correlation matrices from stock returns, constructed minimum spanning trees from these matrices and compared the robustness and evolution of the trees over time. In general we have found the MSTs constructed using the rank correlations (Spearman and Kendall’s ) to be more robust than those constructed with the Pearson correlation. They tend to change less (notably during times of market stress), have edges that are maintained for a longer time period, and tend to have slightly more of the same edges selected when the reliability of the trees is tested using a circular bootstrap. The different methods do however show strong agreement on the hubs in the trees, but tend to disagree on the structure of the peripheries.

The rank MSTs in general show broad agreement on the mean centrality of each sector, but the Pearson MSTs show a significant difference in which sectors they regard as important. The agreement using degree centrality is higher than using betweenness centrality, indicating that companies tend to be found in different places on the trees in the Pearson MSTs compared to the rank ones.

Despite this, the trees tend to have a similar topology, irrelevant of coefficient and this topology tends to vary in a similar way over time for all three methods.

Finally we use a bootstrap to test the consistency of the edges selected in the MSTs. We find the rank MSTs select slightly more of the same edges than the Spearman MSTs, but the difference is not large, and that there is only a weak relationship between the strength of an edge and its likelihood of being selected. In all of the trees intrasector edges are more likely to be selected than intersector edges, and rank based trees tend to select slightly more intrasector edges, but again this difference is small.

This shows that the heavy tails of financial returns have a big effect on the construction of Pearson correlation based MSTs, and this is something to be aware of when trying to draw conclusions from these trees. It therefore may be worth also constructing MSTs using different correlation coefficients for any problem to see if the conclusions drawn are valid for both sets. Generally the Spearman and Kendall’s correlation coefficients tend to give similar results, indicating that if computational resources are constrained then calculating the Spearman correlation is sufficient. Future work could proceed in several directions. A comparison of mutual information MSTs to these correlation MSTs to see how they differ could be interesting, or exploring different filtration models, for instance the Planar Maximally Filtered Graph. Alternatively these comparisons could be performed with returns data from other countries or assets, perhaps from data that is highly correlated and volatile, for instance for returns from cryptocurrencies or developing nations.


TM Acknowledges PhD studentship funding from the School of Electronics and Computer Science, University of Southampton;


  • T. Aste, W. Shaw, and T. D. Matteo (2010) Correlation structure and dynamics in volatile markets. New Journal of Physics 12 (8), pp. 085009. External Links: Document Cited by: item 5.
  • M. Bastian, S. Heymann, and M. Jacomy (2009)

    Gephi: an open source software for exploring and manipulating networks

    In Third international AAAI conference on weblogs and social media, Cited by: §2.
  • V. Boginski, S. Butenko, and P. M. Pardalos (2005) Statistical analysis of financial networks. Computational Statistics & Data Analysis 48 (2), pp. 431 – 443. External Links: Document Cited by: §1.
  • G. Bonanno, G. Caldarelli, F. Lillo, and R. N. Mantegna (2003) Topology of correlation-based minimal spanning trees in real and model markets. Phys. Rev. E 68, pp. 046130. External Links: Document, Link Cited by: item 3.
  • R. Coelho, S. Hutzler, P. Repetowicz, and P. Richmond (2007) Sector analysis for a ftse portfolio of stocks. Physica A: Statistical Mechanics and its Applications 373, pp. 615 – 626. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • P. Coletti (2016) Comparing minimum spanning trees of the italian stock market using returns and volumes. Physica A: Statistical Mechanics and its Applications 463, pp. 246 – 261. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • R. Cont (2001) Empirical properties of asset returns: stylized facts and statistical issues. Quantitative Finance 1 (2), pp. 223–236. External Links: Document, Link, Cited by: §1.
  • S. Drożdż, F. Grümmer, A. Górski, F. Ruf, and J. Speth (2000) Dynamics of competition between collectivity and noise in the stock market. Physica A: Statistical Mechanics and its Applications 287 (3), pp. 440 – 449. External Links: ISSN 0378-4371, Document, Link Cited by: §3.1, §3.2.
  • P. Fiedor (2014) Information-theoretic approach to lead-lag effect on financial markets. The European Physical Journal B 87 (8), pp. 168. Cited by: §1.
  • X. Guo, H. Zhang, and T. Tian (2018) Development of stock correlation networks using mutual information and financial big data. PLOS ONE 13 (4), pp. 1–16. Cited by: §1.
  • A. A. Hagberg, D. A. Schult, and P. J. Swart (2008) Exploring network structure, dynamics, and function using networkx. In Proceedings of the 7th Python in Science Conference, Pasadena, CA USA, pp. 11 – 15. External Links: Document Cited by: §2.
  • J. D. Hunter (2007) Matplotlib: a 2d graphics environment. Computing in Science Engineering 9 (3), pp. 90–95. External Links: Document, ISSN 1521-9615 Cited by: §2.
  • A. Hüttner, J. Mai, and S. Mineo (2018) Portfolio selection based on graphs: does it align with markowitz-optimal portfolios?. Dependence Modeling 6 (1), pp. 63–87. Cited by: item 4.
  • W. Jung, S. Chae, J. Yang, and H. Moon (2006) Characteristics of the korean stock market correlations. Physica A: Statistical Mechanics and its Applications 361 (1), pp. 263 – 271. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • W. Jung, O. Kwon, F. Wang, T. Kaizoji, H. Moon, and H. E. Stanley (2008) Group dynamics of the japanese market. Physica A: Statistical Mechanics and its Applications 387 (2), pp. 537 – 542. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • H. Kaya (2013) Eccentricity in asset management. Available at SSRN 2350429. Cited by: item 4.
  • D. Y. Kenett, M. Tumminello, A. Madi, G. Gur-Gershgoren, R. N. Mantegna, and E. Ben-Jacob (2010) Dominating clasp of the financial sector revealed by partial correlation analysis of the stock market. PLOS ONE 5 (12), pp. 1–14. External Links: Document Cited by: §1.
  • A. Kocheturov, M. Batsyn, and P. M. Pardalos (2014) Dynamics of cluster structures in a financial market network. Physica A: Statistical Mechanics and its Applications 413, pp. 523 – 533. External Links: ISSN 0378-4371, Document, Link Cited by: §3.2.
  • L. Laloux, P. Cizeau, M. Potters, and J. Bouchaud (2000) Random matrix theory and financial correlations. International Journal of Theoretical and Applied Finance 3 (03), pp. 391–397. Cited by: §1, §3.1.
  • R. N. Mantegna (1999) Hierarchical structure in financial markets. The European Physical Journal B - Condensed Matter and Complex Systems 11 (1), pp. 193–197. Cited by: item 1, §1, §1, §3.2.
  • W. McKinney (2010) Data structures for statistical computing in python. In Proceedings of the 9th Python in Science Conference, pp. 51 – 56. Cited by: §2.
  • S. Miccichè, G. Bonanno, F. Lillo, and R. N. Mantegna (2003) Degree stability of a minimum spanning tree of price return and volatility. Physica A: Statistical Mechanics and its Applications 324 (1), pp. 66 – 73. Note: Proceedings of the International Econophysics Conference External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • T. Millington and M. Niranjan (2019) Quantifying influence in financial markets via partial correlation network inference. In 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), Vol. , pp. 306–311. External Links: Document, ISSN Cited by: §1.
  • T. Millington and M. Niranjan (2020) Partial correlation financial networks. Applied Network Science 5 (1), pp. 11. External Links: Document, ISBN 2364-8228, Link Cited by: §1.
  • F. Musciotto, L. Marotta, S. Miccichè, and R.N. Mantegna (2018) Bootstrap validation of links of a minimum spanning tree. Physica A: Statistical Mechanics and its Applications 512, pp. 1032 – 1043. External Links: ISSN 0378-4371, Document, Link Cited by: §3.5.
  • N. Musmeci, V. Nicosia, T. Aste, T. Di Matteo, and V. Latora (2017) The multiplex dependency structure of financial markets. Complexity 2017. Cited by: §1.
  • A. Namaki, A.H. Shirazi, R. Raei, and G.R. Jafari (2011) Network analysis of a financial market based on genuine correlation and threshold method. Physica A: Statistical Mechanics and its Applications 390 (21), pp. 3835 – 3841. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • T. E. Oliphant (2006) A guide to numpy. Vol. 1. Cited by: §2.
  • J. Onnela, A. Chakraborti, K. Kaski, J. Kertesz, and A. Kanto (2003a) Dynamics of market correlations: taxonomy and portfolio analysis. Physical Review E 68 (5), pp. 056110. Cited by: item 2, item 3.
  • J. Onnela, K. Kaski, and J. Kertész (2004) Clustering and information in correlation based financial networks. The European Physical Journal B 38 (2), pp. 353–362. Cited by: §1, §1.
  • J.-P. Onnela, A. Chakraborti, K. Kaski, and J. Kertész (2003b) Dynamic asset trees and black monday. Physica A: Statistical Mechanics and its Applications 324 (1), pp. 247 – 252. Note: Proceedings of the International Econophysics Conference Cited by: item 4.
  • G. Peralta and A. Zareei (2016) A network approach to portfolio selection. Journal of Empirical Finance 38, pp. 157 – 180. Cited by: item 4.
  • V. Plerou, P. Gopikrishnan, B. Rosenow, L. A. N. Amaral, T. Guhr, and H. E. Stanley (2002) Random matrix approach to cross correlations in financial data. Physical Review E 65 (6), pp. 066126. Cited by: §1, §3.1.
  • F. Pozzi, T. Di Matteo, and T. Aste (2012) Exponential smoothing weighted correlations. The European Physical Journal B 85 (6), pp. 175. External Links: ISSN 1434-6036, Document, Link Cited by: §1.
  • F. Pozzi, T. Di Matteo, and T. Aste (2013) Spread of risk across financial markets: better to invest in the peripheries. Scientific reports 3, pp. 1665. Cited by: item 4.
  • S. Seabold and J. Perktold (2010) Statsmodels: econometric and statistical modeling with python. In 9th Python in Science Conference, Cited by: §2.
  • K. Sheppard, S. Khrapov, G. Lipták, R. Capellini, esvhd, Hugle, JPN, X. RENE-CORAIL, M. E. Rose, and jbrockmendel (2019) Bashtage/arch: release 4.8.1 External Links: Document, Link Cited by: §2.
  • O. Shirokikh, G. Pastukhov, V. Boginski, and S. Butenko (2013) Computational study of the US stock market evolution: a rank correlation-based network model. Computational Management Science 10 (2), pp. 81–103. External Links: Document, Link Cited by: §1.
  • J. Y. Song, W. Chang, and J. W. Song (2019) Cluster analysis on the structure of the cryptocurrency market via bitcoin–ethereum filtering. Physica A: Statistical Mechanics and its Applications 527, pp. 121339. External Links: ISSN 0378-4371, Document, Link Cited by: §1.
  • D. Stosic, D. Stosic, T. B. Ludermir, and T. Stosic (2018) Collective behavior of cryptocurrency price changes. Physica A: Statistical Mechanics and its Applications 507, pp. 499–509. Cited by: §1.
  • P. Tewarie, A. Hillebrand, M.M. Schoonheim, B.W. van Dijk, J.J.G. Geurts, F. Barkhof, C.H. Polman, and C.J. Stam (2014) Functional brain network analysis using minimum spanning trees in multiple sclerosis: an meg source-space study. NeuroImage 88, pp. 308 – 318. External Links: ISSN 1053-8119, Document, Link Cited by: §1.
  • P. Tewarie, E. van Dellen, A. Hillebrand, and C.J. Stam (2015) The minimum spanning tree: an unbiased method for brain network analysis. NeuroImage 104, pp. 177 – 188. External Links: ISSN 1053-8119, Document, Link Cited by: §1.
  • R. S. Tsay (2005) Analysis of financial time series. Vol. 543, John Wiley & Sons. Cited by: §1.
  • M. Tumminello, C. Coronnello, F. Lillo, S. Micciche, and R. N. Mantegna (2007)

    Spanning trees and bootstrap reliability estimation in correlation-based networks

    International Journal of Bifurcation and Chaos 17 (07), pp. 2319–2329. Cited by: §3.5, §3.5.
  • M. Tumminello, F. Lillo, and R. N. Mantegna (2010) Correlation, hierarchies, and networks in financial markets. Journal of Economic Behavior & Organization 75 (1), pp. 40 – 58. Note: Transdisciplinary Perspectives on Economic Complexity External Links: ISSN 0167-2681, Document, Link Cited by: §1.
  • N. Vandewalle, F. Brisbois, X. Tordoir, et al. (2001) Non-random topology of stock markets. Quantitative Finance 1 (3), pp. 372–374. Cited by: item 3.
  • Y. Zhang, G. H. T. Lee, J. C. Wong, J. L. Kok, M. Prusty, and S. A. Cheong (2011) Will the us economy recover in 2010? a minimal spanning tree study. Physica A: Statistical Mechanics and its Applications 390 (11), pp. 2020 – 2050. Cited by: item 2.