Networks are one of the most frequently used modelling paradigms for dynamical systems. Investigations towards synchronization phenomena in networks of coupled oscillators have attracted considerable attention, and so has the analysis of chaotic behaviour and corresponding phenomena in networks of dynamical systems to name just a few Kralemann et al. (2014); Pikovsky (2016); Cestnik and Rosenblum (2017); Li et al. (2014). Understanding and characterizing network behaviour has triggered interest in a vast number of disciplines, ranging from optimizing vaccination strategies Clusella et al. (2016) to understanding the functioning or malfunctioning of the human brain Bullmore and Sporns (2009); Pessoa (2014); Petersen and Sporns (2015).
While first principle modelling is feasible in some areas, in others, networks need to be inferred, e.g. from observed data, see, e.g., Guimerà and Sales-Pardo (2009); Levnajić and Pikovsky (2011); Karrer and Newman (2011); Alderisio et al. (2017). This comes with certain challenges ranging from selecting the appropriate nodes or even defining them, to the choice of an appropriate technique to infer the interaction between the nodes. These choices have a strong impact on the resulting network. Here, we discuss another related challenge that originates from the fact that network inference in the Inverse Problem typically relies on statistical methods and selection criteria.
A typical network inference procedure, estimates the connectivity between a-priori specified nodes in a network. If the connectivity measure passes a certain threshold, a link between the corresponding nodes is assumed to be present. The choice of this threshold is arbitrary, but it is intuitively clear that there is a strong correlation between number of links and choice of threshold. Selecting the threshold, not only controls how many links are inferred correctly but also establishes the number of incorrectly determined links. There are two types of errors, (i) a link may be erroneously considered present, this false positive conclusion is referred to as a type I error; (ii) a present link may remain undetected, this false negative conclusion is referred to as a type II error.
In this manuscript, we present an analytical framework that on the network level links the reconstructed network structure contaminated by type I and type II errors to the true underlying one. While the framework is rather general, we used the vertex degree distribution to derive the functional relationship between the reconstructed and true underlying network. This enables us to obtain superior estimates for the vertex degree distribution, a key property of a network Newman (2010). It has been shown that including the vertex degrees into stochastic blockmodels improves their performance for statistical inference of group structure Karrer and Newman (2011). The functional relationship depends on the choice of type I error, type II error and the dimension of the network. We demonstrate the performance of our novel approach in a simulation study.
Ii Materials and Methods
In section II.1, a short introduction to networks is presented; we analyse the influence of type I and type II errors on the network structure, i.e. false positive and false negative conclusions about links. In section II.2 different methods to solve the Inverse Problem are presented. Section II.3 contains a brief description of the generalization to directed networks.
ii.1 Networks change
A network is defined as a set of vertices (or nodes) with links (or edges) between them. To quantify the structure of networks, different characteristics have been introduced Olbrich et al. (2010). Here, we consider two key network characteristics: vertex degree distribution and number of edges. The vertex degree describes the number of links of a node; if the vertex has edges attached, its vertex degree is . The vertex degree distribution is an important property of the entire network.
Networks can be directed or undirected Newman (2010). In an undirected network, connection of to implies the connection of to . In a directed network, this symmetry is broken, therefore if a path from to exists, a path from to does not necessarily exist. In this section, we consider undirected networks. Later [Sec. II.3] a generalization to directed networks is presented.
Consider a network with
nodes and vertex degree distribution defined by the probability function, i.e., is the probability that the degree is , for . Note that the degree of a vertex is between and , since each vertex can be connected to at most remaining vertices.
We analyse the influence of type I and type II errors on the vertex degree distribution of a given network . We call the network detected when type I and type II errors occur and we assume that is the probability of a type I error and that is the probability of a type II error. Therefore, expresses the probability that a link absent in is present in and is the probability that a link present in is no longer present in . Hence, the set of edges of is a combination of true positive links and false positive links of . The vertex degree distribution of is characterised by the probability function .
Consider a vertex and assume it has degree , therefore there are links connected to it and absent links. We evaluate the probability that this vertex has vertex degree in . The vertex degree
is given by the sum of true positive links and false positive links ; additionally, and have to satisfy
The condition described by Eq. (2a) guarantees that the number of false negative links is larger or equal than zero, and smaller or equal than the number of the original true positive links, i.e., . Likewise, the number of false positive links must be non-negative and smaller or equal than the number of the original non-present links, Eq. (2b).
The probability that a vertex has degree in , knowing it has degree in is
The probability of having true positive links, over all possible original true positive links, is
which is the binomial distribution. Since [Eq. (1)], Eq. 4 corresponds to the first part of the first case of Eq. 3. Similarly, the probability of having false positive links is
which is the binomial distribution .
Combining Eqs. (4) and (5), changing variable according to Eq. (1), and considering all possible combinations of and , we obtain the first case of Eq. (3). All the other cases can be derived in the same way following the conditions in Eq. (2).
Applying the law of total probability
we obtain the matrix equation
The matrix depends on , and and has determinant
therefore it is invertible if and only if , see Appendix A for a proof.
Assuming is known, Eq. (7) characterises the influenced of type I and type II errors on the vertex degree distribution, and it allows to find the vertex degree distribution of the network .
ii.2 Inference of networks’ vertex degree distribution
Section II.1 analyses the impact of type I and type II errors on the vertex degree distribution of a given network. Equation (7) allows to obtain from . In this section, we are interested in the inverse problem, i.e., inverting Eq. (7), to infer the original vertex degree distribution from an observed one. When , since the convergence to zero of the determinant of scales like for [Eq. (8)], numerical issues arise for relatively small when inverting the matrix to find through . The cases and are trivial, see Appendix A.
The least squares method is a standard approach to solve problems like Eq. (7). Although the matrix is not singular, for reasonable parameter values for ,
is typically ill-conditioned, therefore the pseudoinverse of the truncated singular value decomposition ofis used.
The singular value decomposition of a matrix is the factorization of the matrix into the product of where is a diagonal matrix and the columns of the matrices and are orthonormal (Yanai et al., 2011). The elements on the diagonal of are called singular values of and they are ordered such that , where is the rank of .
The singular value decomposition is a tool to compute the pseudoinverse of a matrix. If has singular value decomposition , its pseudoinverse is defined as , where is obtained from replacing all the non-zero elements with their reciprocals.
The truncated singular value decomposition is a method for regularization of ill-posed least squares problems (Hansen, 1987). Once the singular value decomposition is found, the matrix is truncated at, e.g., rank such that only the first singular values are considered; this matrix is usually called . More precisely, is a diagonal matrix with elements , with . The truncated diagonal matrix is used to find an approximation of the matrix using its decomposition, i.e., . The optimal value for has been studied in (Frank and Buhmann, 2011; Gavish and Donoho, 2014). The matrix is the closest approximation of of rank , (Hansen, 1987). Using , we calculate the pseudoinverse of , i.e., , and we solve Eq. (7) resulting in
ii.3 Generalization for directed networks
For directed networks the vertex degree is characterised by the vertex in-degree and the vertex out-degree, Newman (2010). Usually, in a directed network the vertex degree is the sum of the vertex in-degree and the vertex out-degree.
Both the in-degree and the out-degree of a vertex are numbers between and , if is the number of vertices of the network. Therefore, the analysis shown in Section II.1 and II.2 remains valid if either the vertex in-degree or the vertex out-degree are considered instead of the vertex degree.
An undirected network with nodes has at most edges. A network with nodes has at most directed edges; a generalization for other characteristics is likely more complicated, and therefore requires a more in-depth analysis.
Iii Results and Discussions
To demonstrate the abilities as well as limitations, the analysis presented in Section II is applied to some typical simulated networks. We like to highlight that our approach is derived analytically; simulation studies are predominantly needed to demonstrate its applicability in real-world examples and to check for numerical issues, etc. There might be practical issues, e.g., due to the dimension of the network, and with the aim to show how these challenges can be overcome we present a simulation study to explore the concrete applicability of our method.
We study 5 network topologies that present different characteristics so to have a spectrum of networks as wide as possible to which we apply our analysis. Namely, we consider Erdős-Rényi (also called random), Small-World, Scale-Free networks, a three-dimensional grid, and a network of randomly connected communities, Newman (2010). We vary the probabilities and of type I and type II errors in the range mimicking a typical analysis method that has high sensitivity and high specificity. Nevertheless, both lower and higher values for and can be chosen and the results obtained are qualitatively the same as the ones presented below.
Consider a random network with nodes and a probability of a connection of . The vertex degree has binomial distribution . Adding and removing links with probabilities and results in a new network . The vertex degree distribution of is calculated empirically by counting the vertices’ degrees. Applying the procedure explained above, the vertex degree distribution of the original network is estimated. Figure 1 shows the results using the cut-off for the truncated singular value decomposition method of , i.e., contains only singular values greater than . The choice of is motivated by smoothness and regularity of the solution obtained.
Figure 1 shows the histogram of the degrees of the vertices of the original network , the density of the detected network , the reconstructed vertex degree distribution of the original network resulting from Eq. (9), and the result when a non-negative constraint is applied to the truncated singular value decomposition to avoid that numerical issues result in negative solutions. More precisely, we use Matlab function with lower bound condition ; this function implements the trust region reflective algorithm Coleman and Li (1994); Branch et al. (1999). The density of is estimated by
its empirical distribution, and this is used to infer the original network.
Figure 2 shows the result when the original network is a Small-World network. It is built from the regular network of nodes, vertex degree , and probability of rewiring . The network is obtained by adding and removing links at random with probabilities and respectively. The cut-off for the truncated singular value decomposition method is .
Figure 2 shows the histogram of the degrees of the vertices of the original network , the density of the detected network , the reconstructed vertex degree distribution, and the result when a non-negative constraint is applied to the truncated singular value decomposition.
Figure 3 shows the result when the original network is a Scale-Free network. It is built using a preferential attachment model for network growth. At each step a vertex, with a link attached to it, is added. The probability that the new vertex attaches to a given old one is proportional to its vertex degree. This procedure is repeated until the network has nodes. The network is obtained by adding and removing links at random with probabilities and respectively. The cut-off for the truncated singular value decomposition method is .
Figure 3 shows the histogram of the degrees of the vertices of the original network , the density of the detected network , the solution of Eq. (9), and the result when a non-negative constraint is applied to the truncated singular value decomposition.
Another example we apply our method to, is when the original network is a three-dimensional grid ; note that has nodes. The network is obtained by adding and removing links at random with probabilities and respectively. The cut-off for the truncated singular value decomposition method is . Figure 4 shows the histogram of the degrees of the vertices of the original network , the density of the detected network , the solution of Eq. (9), and the result when a non-negative constraint is applied to the truncated singular value decomposition.
Finally, we investigate the case when is a network of three randomly connected communities. It is built by constructing three Erdős-Rényi networks, with probability of connection , , and , and each with nodes. Then, nodes from different communities are connected with probability . The network is obtained by adding and removing links at random with probabilities and respectively. The cut-off for the truncated singular value decomposition method is . Figure 5 shows the histogram of the degrees of the vertices of the original network , the density of the detected network , the solution of Eq. (9), and the result when a non-negative constraint is applied to the truncated singular value decomposition.
Another interesting aspect is the influence of type I and type II errors and the proposed method on the reconstruction of individual nodes and not just the correct distribution. This is particularly relevant for nodes that have a degree much higher than average, so-called hubs.
In the Scale-Free example, Fig. 3, the detected distribution appears to be smoother than the original, implying that a hub might have been converted to a non-hub. Analysing this in more detail, there is convincing evidence that this is not the case - hubs are correctly identified as hubs.
Consider a node that has degree in that has nodes. Due to type I and type II errors, this node in has degree
, a random variable with distribution shown in Eq. (3). Taking realisations of this random variable, and inverting the process using Eq. (9), allows us to compare individual degrees for a given node of the true network with the reconstructed one. We consider a network with nodes, a node with degree , probabilities of type I and type II errors of and , respectively, and we simulate realisations of the random variable described above. Figure 6 shows the reconstruction of the degree of using these realisations. The result does not only show an improvement from the detected degrees , but also illustrates the high accuracy of the reconstruction method.
Figure 7 shows the reconstruction of various degrees, i.e., from to in steps of , using the same parameters , , , and realisations each. This again demonstrates that our method reliable reconstructs the correct degree for this individual node. Further simulations, not presented here, varying and between and , show qualitatively the same results. In every case, the reconstruction is very robust, and this suggests that it is extremely unlikely that a hub is reconstructed as a non-hub. Moreover, the reconstruction works correctly not only on the general distribution, but also when it is applied to single nodes.
iii.1 Robustness of reconstruction
As stated above, our reconstruction method assumes the probabilities of type I and type II errors to be known a priori. While the type I error is controlled by statistical methods, the type II error must be inferred or reasonable assumptions from simulations, or prior studies, about the type II error must be available. To show the impact of violations of this and thereby the robustness of our method, we analyse the performance of the reconstruction when perturbations on and are introduced.
Figures 8-11 demonstrate the robustness of our approach for various examples. Figures 8-9 are used to show robustness with respect to , while Figs. 10-11 show the robustness with respect to . The perturbations are quantified in percentage using the parameter , e.g. the perturbations of are expressed by . Note that, since , the conditions for the perturbations are ; namely, if we call the perturbed , then we have
Negative values for represent underestimated values for and positive overestimated values for . The same argument is used for the perturbation of .
Figure 8 shows the reconstruction of a Scale-Free network with nodes for the true value of , and also for various values of deviating up to from the true value, i.e., . The cut-off for the truncated singular value decomposition method is and the probability of type I error is , assuming to control the family-wise error rate at this value, i.e., the probability of making at least one type I error; it is beyond the scope of this manuscript to discuss cases in which the technique selected to reconstruct the network violates this assumption - we will however estimate the results for different deviations from the true used to generate the plots to investigate its robustness. Figure 8 shows that our approach is robust to rather large perturbations of , in both negative and positive directions. Up to , the bias of the reconstruction is negligible; only if or more deviates the reconstruction significantly from the true one, although it still performs better than the naïve approach of trusting the identified network structure.
Figure 9 shows the reconstruction of a random network with nodes and probability of a connection of for the true value of , and also for various values of deviating up to from the true value of . The cut-off for the truncated singular value decomposition method is and the probability of type I error is . Also in this case, the method is robust to large perturbations of , in both negative and positive directions. A deviation of more than is needed for the method to fail and not to have an improvement over the naïve approach.
Figure 10 shows the reconstruction of a random network with nodes and probability of a connection of for the true value of , and deviating up to . The cut-off for the truncated singular value decomposition method is and the probability of type II error is . Figure 10 shows that the method is affected by relatively large perturbations of . Namely, for and , the reconstructions deviate significantly from the true one. The reason is that sparse networks are susceptible to perturbation of type I error. Figure 11 shows the reconstruction of a denser network, i.e., a random network with probability of a connection of , for the same true values of and . In this case, a deviation of or more is needed for the method to fail. Comparing Figs. 10 and 11, we can conclude that dense networks are more robust to perturbations of type I error than sparse networks. This is intuitively motivated by the fact that the type I error affects links that are not present in the network, and therefore it has a bigger influence on a sparse network.
The above results demonstrated that a rough estimate for and is sufficient to get an accurate reconstruction; the method is robust to relatively large perturbations of these two errors. Rough estimates of these parameters are typically available from simulation studies or prior knowledge about the system. Note again that the role of and are different;
is often controlled and can be obtained from known statistics of the techniques under the null hypothesis;is more difficult as the true alternative would need to be known. Given the above simulations, our algorithm is more robust with respect to than , which aligns with the different role of these two errors. As mentioned at the beginning of Section III, we vary and in the range mimicking a typical analysis method that has high sensitivity and high specificity. Choosing either lower or higher values for the true and does not affect the general qualitatively result of the analysis, but it changes the range of perturbation that leads to the failure of the reconstruction method.
We explore the impact of false positive and false negative conclusions about the presence or absence of links on the vertex degree distribution of a network. Using an analytical approach, we investigate this dependence on the dimension of the network and the probabilities of type I and type II errors. Equation (7) describes the density of the vertex degree distribution of the biased network and thus allows to calculate the influence of false positive and false negative conclusions about links on any kind of network, assuming the probabilities of type I and type II errors are known.
In the inverse problem, the aim is to reconstruct the original network. Equation (9) enables us to calculate analytically the vertex degree distribution of the original network if the biased one and the probabilities of type I and type II errors are given. When the dimension of the network is relatively large, numerical issues arise and consequently the truncated singular value decomposition is used to calculate the original network vertex degree distribution. Numerical simulations show that the vertex degree distribution is correctly recovered in all the cases discussed; the cases presented are designed to cover a variety of network topologies and therefore degree distributions.
The outcomes of this manuscript are general results that enable to reconstruct analytically the vertex degree distribution of any network. The analytic formula [Eq. (9)] that allows to find the original vertex degree distribution depends only on the detected vertex degree distribution and on the probabilities of type I and type II errors. This method is a powerful tool since the vertex degree distribution is a key characteristic of networks. Moreover, we have actually shown that this method can be used to reconstruct individual node degrees to a very high accuracy. This should positively impact on various measures that can be derived from the networks. Our proposed method should outperform standard approaches in terms of betweenness centrality, identification of hubs, and other network characteristics. This should be rigorously assessed in future research.
A limitation of this work is the assumption that the probabilities of type I and type II errors are known a priori. Nevertheless, we show that the method is robust to relatively large perturbations of these two errors. Therefore, wrong estimates of type I and type II errors, within certain bounds, do not cause the reconstruction of be rendered invalid. We like to emphasise again that in application the type I error is typically controlled, while an estimate for the type II error can only be obtained through prior experiments/knowledge or simulation studies. As shown in various simulations, our reconstruction method is robust to considerable deviations in , which supports the usefulness of our technique over and above providing deeper insights into the role of these errors in network reconstruction; our approach is promising for real-world applications. Note though that it is always advisable to utilise simulation studies to characterise the advantageous and limitations in a concrete application at hand. Further analyses should study possible statistical approaches to infer these parameters employing Bayesian approaches or simulation studies. We recommend performing the latter to get an estimate of the type II error in particular.
Future studies should investigate the influence of type I and type II errors on other network characteristics, e.g. the number of edges, the global clustering coefficient, and the efficiency. As a consequence, more information about the original network can be found and, therefore, combining them all a better reconstruction of the network can be achieved.
Acknowledgements.The authors thank Dr. Daniel Vogel for helpful comments and discussions. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 642563. The authors declare no competing financial interests.
Appendix A Determinant of the matrix A
In this appendix we prove that the matrix has determinant
To achieve this we have to prove some intermediate steps.
First, we write Eq. (3) in a more compact form as
Since the element is defined as the probability , we can write
for and real numbers .
Proposition 1 (Limit cases).
When , the Eq. (12) becomes
Note that does not depend on but only on
, therefore in each line all the elements are identical, i.e., it is a multiple of vector; hence, all the lines are linear depend and then the determinant of is .
If then the matrix is the identity and therefore the determinant is . While if then the matrix is anti-diagonal with all elements equal to one, then the determinant is .
If , Eq. (12) can be formulated as
Note that since the case has been already considered. The matrix is upper triangular and therefore the determinant is the product of the elements on the diagonal , i.e.,
If , Eq. (12) can be formulated as
The matrix has all zeros above the anti-diagonal and therefore the determinant is the product of the elements on the anti-diagonal and sign given by , i.e.,
If , Eq. (12) can be formulated as
The matrix is lower triangular and therefore the determinant is the product of the elements on the diagonal , i.e.,
If , Eq. (12) can be formulated as
The matrix has all zeros below the anti-diagonal and therefore the determinant is the product of the elements on the anti-diagonal and sign given by , i.e.,
Future calculations result easier if the transpose of matrix is considered. Considering instead of does not affect the calculation of the determinant since it is in general true that .
Proposition 2 (Transformations).
Given the matrix A, defined by Eq. (12), let’s call the transpose of of dimension . Let be and . We call the matrix with elements
where are the elements of the matrix . Then, we prove that
To verify Eq. (18), we have proved that the identity i.e.,
are always valid. To achieve this, we have split the calculations into cases, i.e.,
For each condition the identity has been proved. ∎
The matrix A, defined by Eq. (12), has determinant
For or , Proposition 1 proves the theorem. Assume and . Since a matrix and its transposed have the same determinant, we proceed considering the matrix proving a proof by induction.
The base of induction is ; in this case the matrix is
and it has determinant .
The inductive step consists in assuming that , i.e., the inductive hypothesis for dimension , and proving the statement for dimension .
Since and , we can apply Proposition 2. The transformations defined by Eq. (17) guarantees that the matrix has the same determinant as , and according to Eq. (18), we can express the determinant as . We can now apply the inductive hypothesis, therefore
that concludes the proof of the theorem. ∎
- Kralemann et al. (2014) B. Kralemann, A. Pikovsky, and M. Rosenblum, New J. Phys. 16, 085013 (2014).
- Pikovsky (2016) A. Pikovsky, Phys. Rev. E 93, 062313 (2016).
- Cestnik and Rosenblum (2017) R. Cestnik and M. Rosenblum, Phys. Rev. E 96, 012209 (2017).
- Li et al. (2014) S. Li, F. Li, W. Liu, and M. Zhan, Physica A: Statistical Mechanics and its Applications 404, 118 (2014).
- Clusella et al. (2016) P. Clusella, P. Grassberger, F. J. Pérez-Reche, and A. Politi, Phys. Rev. Lett. 117, 208301 (2016).
- Bullmore and Sporns (2009) E. Bullmore and O. Sporns, Nat. Rev. Neurosci. 10, 186 (2009).
- Pessoa (2014) L. Pessoa, Phys. Life Rev. 11, 400 (2014), arXiv:1403.7151 .
- Petersen and Sporns (2015) S. E. Petersen and O. Sporns, Neuron 88, 207 (2015).
- Guimerà and Sales-Pardo (2009) R. Guimerà and M. Sales-Pardo, Proceedings of the National Academy of Sciences 106, 22073 (2009), arXiv:1004.4791 .
- Levnajić and Pikovsky (2011) Z. Levnajić and A. Pikovsky, Phys. Rev. Lett. 107, 034101 (2011).
- Karrer and Newman (2011) B. Karrer and M. E. J. Newman, Phys. Rev. E 83, 016107 (2011).
- Alderisio et al. (2017) F. Alderisio, G. Fiore, and M. di Bernardo, Phys. Rev. E 95, 042302 (2017).
- Newman (2010) M. Newman, Networks: An Introduction (Oxford University Press, Inc., New York, NY, USA, 2010).
- Olbrich et al. (2010) E. Olbrich, T. Kahle, N. Bertschinger, N. Ay, and J. Jost, Eur. Phys. J. B 77, 239 (2010).
- Yanai et al. (2011) H. Yanai, K. Takeuchi, and Y. Takane, Projection Matrices, Generalized Inverse Matrices, and Singular Value Decomposition (Springer, 2011).
- Hansen (1987) P. C. Hansen, BIT Numerical Mathematics 27, 534 (1987).
- Frank and Buhmann (2011) M. Frank and J. M. Buhmann, (2011), arXiv:arXiv:1102.3176v3 .
- Gavish and Donoho (2014) M. Gavish and D. L. Donoho, IEEE Transactions on Information Theory 60, 5040 (2014).
- Coleman and Li (1994) T. F. Coleman and Y. Li, Mathematical Programming 67, 189 (1994).
- Branch et al. (1999) M. A. Branch, T. F. Coleman, and Y. Li, SIAM J. Sci. Comput. 21, 1 (1999).