Dynamical systems on directed hyper-graphs

02/25/2022
by   Mauro Faccin, et al.
0

Networks and graphs provide a simple but effective model to a vast set of systems which building blocks interact throughout pairwise interactions. Such models fail to describe all those systems which building blocks interact at a higher order. Higher order graphs provide us the right tools for the task. We analyze the interplay between the structure of a directed hyper-graph and a dynamical system evolving on it. We connect the dynamical system on the h to a corresponding random walk on an effective graph. Measures of hyper-dynamics correspond to similar measures on the effective dynamics. We can use simple and well tested algorithms on the effective graph.

READ FULL TEXT VIEW PDF

Authors

page 5

02/06/2021

Understanding Higher-order Structures in Evolving Graphs: A Simplicial Complex based Kernel Estimation Approach

Dynamic graphs are rife with higher-order interactions, such as co-autho...
04/20/2021

What are higher-order networks?

Modeling complex systems and data using the language of graphs and netwo...
10/23/2020

Towards Dynamic-Point Systems on Metric Graphs with Longest Stabilization Time

A dynamical system of points on a metric graph is a discrete version of ...
02/16/2015

Rewriting Higher-Order Stack Trees

Higher-order pushdown systems and ground tree rewriting systems can be s...
07/15/2020

Motifs for processes on networks

The study of motifs in networks can help researchers uncover links betwe...
07/09/2018

Learning Functions in Large Networks requires Modularity and produces Multi-Agent Dynamics

Networks are abundant in biological systems. Small sized over-represente...
06/03/2020

dynoNet: a neural network architecture for learning dynamical systems

This paper introduces a network architecture, called dynoNet, utilizing ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Studying complex systems often requires disaggregating those systems into simpler parts that interact with each other.

Network science often name those base constituents nodes and the pairwise interaction between them edges. Unfortunately, in some cases pairwise interactions fall short to describe the initial system and one needs to follow a different route. Within the different methods introduced to tackle such problem, simplicial complexes introduce the notion of nodes that interact through simplices [bianconi2016simplicial, petri2018simplicial] such as a segment, a triangle, a tetrahedron etc. Hyper-graphs, on the other hand, define interaction beyond pairwise through the definition of hyper-edges, edges that affect multiple nodes at once [zhou2007learning_hypergraphs, Schaub_Signal_proc_2021].

When dynamical processes are coupled to the linking pattern of interacting nodes, the selection of a model to analyze the original system depends on the double choice of the topological and the dynamical models. Further, the necessity to analyze such system requires the development of new tools that extend and complete the measures developed along the years in the cases of simple pair-wise graphs and adapt them to the selected model.

In this work we leverage the results on directed hyper-graphs [gallo1993directed, tran2019dir_hypergraph_pagerank] and recent finding characterizing dynamical systems on undirected hyper-graphs [carletti2020stability, Eriksson2021flowbased]

. We will show that, when the conditions are satisfied and the appropriate dynamics are defined on the hyper-graph, one can compute a transition matrix representing the probability to transition between any two nodes of the hyper-graph. Such transition matrix defines

effective dynamics on a family of (pairwise) graphs that will microscopically reproduce the same (dynamical) behavior of the original higher-order graph, at the steady state. One may expect that measures that are function of the dynamical system, should return the same values on the hyper-graph dynamics and its pairwise effective dynamical counterpart. Finally, we describe some examples to illustrate these findings.

Ii Directed Hs and dynamical systems

While the selection of the topological and the dynamical models depend on each other, in the following, for simplicity, we describe them independently.

ii.1 Topology

A h is defined by a set of nodes that interact through a set of hyper-edges . A hyper-edge represents the interaction between two or more nodes, the interaction can be directed. When the hyper-graph is directed, the nodes involved in the interaction are divided in two (possibly overlapping) subsets: the source of the interaction called tail of the hyper-edge; and the target of the interaction called head (see Figure 1). The nomenclature, tail and head, represents the directionality of the interaction through the figure of the arrow.

Figure 1: A hyper-edge. Node is the tail and represents the entry point of the hyper-edge. Nodes and are the exit points of the dynamics and are termed head.

The boolean tail () and head () matrices are defined as follows:

This construction is reminiscent of the chemical reaction hyper-graphs [Mulas_2021] where incoming and outgoing interactions are defined by positive and negative values.

Hyper-edge weights can be introduced as a weighting factor . In some cases, as in the following, it may be useful to define hyper-edges weights as functions of the corresponding hyper-edge characteristics (such as the number of nodes involved).

We can introduce an interaction matrix .

(1)

where is the square weight matrix with hyper-edge weights on the diagonal.

Forward hyper-edges

In the following we can suppose that all hyper-edges have tails of size one, the so-called forward hyper-edges. Otherwise, hyper-edges with larger tails can decompose to a set of forward hyper-edges each with the tail on a node of the former hyper-edge and the same head.

In this framework, all columns of the tail matrix have exactly one non-zero element (). We will refer to the size of the hyper-edge () as the number of nodes included into its head: . When all the hyper-edges have size one (), the hyper-graph reduces to a normal graph with pair-wise interactions. A hyper-graph is symmetric when for any hyper-edge , there exists other hyper-edges with tail on each of its head nodes and head on the other involved nodes.

Hyper-edge weights as function of the size.

It may be useful to define the weight on the hyper-edge as a function of the hyper-edge size, in particular:

(2)

where is a biasing parameter. For positive values of the latter, hyper-edges with large heads have higher weights, and vice versa for negative values.

ii.2 Dynamics on the hyper-graph

A dynamical system that leverage the hyper-graph connectivity can be defined in a number of ways. In the following we describe a family of linear dynamics on a hyper-graph, as defined in [carletti2020randomwalks] for the symmetric case.

A dynamical system linked to the hyper-graph can be interpreted as a special case of the (hyper) graph signal processing [Schaub_Signal_proc_2021]

, where the signal vector is constrained to be normalized and the shift operator is a right stochastic matrix.

Consider, in this case, a dynamical system evolving on a directed hyper-graph , as a generalization of the classical random walk on a simple graph. At each time step the walker, sitting on node , will perform the following steps:

  1. choose a neighboring hyper-edge which tail include node with a probability proportional to the hyper-edge weight,

  2. traverse the hyper-edge to one of the nodes of its head,

Hence, the biasing parameter will influence the dynamics warping the walker toward hyper-edges with larger or smaller heads. In particular, with the dynamics will select each tail with equal probability [zhou2007learning_hypergraphs] while with they will select each tail with probability proportional to the respective hyper-edge size. The transition probability between two nodes and can be written as (with a simple normalization):

(3)

where is a bias parameter, and the interaction matrix of Eq. 1, with the hyper-edge weights as in Eq.2. The probabilities in Eq.3 determine the entries of the right stochastic transition matrix . In particular:

(4)

where is the vector of all ones.

Assuming ergodic dynamics, one can compute a steady state which encodes the probability of the walker to visit a given node of the hyper-graph. The steady state

is the dominant eigenvector of the transition matrix

that corresponds to its unit eigenvalue.

The probability of leaving node and reaching node at the following time step is . This defines the probability matrix:

(5)

The effective graph.

At this point, an interesting question is: can we find an effective graph which, coupled with a random walk, will microscopically reproduce the dynamical behavior of the original system?

The answer to this requires a distinction.

In the simpler case of a symmetric hyper-graph, one would expect a symmetric effective graph, this implies that the corresponding adjacency matrix would be proportional to the probability matrix of Eq.5. In particular the interaction matrix is in this case a suitable representative of all possible effective adjacency matrices, which can all be written as (with a positive multiplicative factor and the norm).

Unfortunately, this is not the case in for a directed hyper-graph. There exists, in fact, a (infinite) family of graphs that correspond to a given transition matrix. Since we cannot assume the symmetry of any effective graph and the out-degree sequence cannot be imposed, any matrix of the form is a suitable adjacency matrix. In particular, in this case, the interaction matrix and the probability matrix together with the transition matrix itself, are only few representatives of that family.

The clique expansion.

Often, in the literature, one can find approaches involving the clique expansion of (symmetric) hyper-graphs. Here, a hyper-edge is replaced by a clique of (pairwise) links between all the nodes involved in the original higher order interaction. This approach assumes that the dynamics can freely flow between any pair of nodes involved in the interaction.

Considering the analysis conducted so far, for a symmetric hyper-graph, clique expansion use the interaction matrix of Eq.1 as adjacency matrix of the effective graph, fixing . This choice is assuming, thus, a precise (linear) dynamical model, in particular with , where a random walker will choose neighboring hyper-edges according (proportionally) to their head size.

Iii Measuring dynamics

One may expect that measures that are mere function of the dynamical system return the same result when applied to a dynamical system on a hyper-graphs or to a pairwise graph, when those display the same dynamical behavior. Measures of the transition matrix (Eq. 3) or the probability matrix (Eq. 5) that characterize the dynamical system will depend on the value of the biasing parameter . However, those measures involve pair-wise matrices and provide a basis for a straightforward extension of similar measures on simple graphs.

iii.1 Node ranking

For a simple graph, the eigenvector centrality of node represents its connection degree to other highly ranked nodes. The eigenvector equation computes the centrality vector in vector notation:

(6)

where is the largest eigenvalue of the adjacency matrix .

Figure 2: Example of a hyper-graph of 5 nodes with 6 directed hyper-edges (listed on the left). The eigencentrality node ranking change depending on the value of the bias parameter (table on the right). The probability corresponding to 3 values of are represented on the lower right part as graphs. One can notice, in particular, that node 0 and node 1 change their relative order within the ranking.

In the hyper-graph case, we have no access to the adjacency matrix, but a good candidate for that role is the probability matrix of Eq. 5:

(7)

In Figure 2 we present a didactic example of a hyper-graph of 5 nodes and 6 hyper-edges. In this graph, hyper-edges of size one point to node 0, while hyper-edges of size 3 point mostly to node 1. Note how the ranking of nodes according to the eigencentrality defined as in Eq. 7 depends on the value of the bias parameter . For negative values of the dynamics follow preferentially small edges, hence node 0 has the highest rank. For positive values of the highest ranked node is node 1.

Pagerank [brin1998pagerank] is a similar node ranking algorithm designed to rank web-pages according to the user visiting behavior. The algorithm models the browsing behavior of the user as a random walk with teleportation and the ranking value corresponds to the visiting probability of each node (the dominant right eigenvector of the transfer matrix, comprising the teleportation term). Extending the pagerank to the hyper-graph framework is straightforward as merely a function of the transfer matrix.

Table 1 shows the ranking of the hyper-graph nodes as in the hyper-graph of Figure 2. Notice a ranking reordering similar to that from eigencentrality when the bias parameter changes its value.

ranking prob ranking prob ranking prob
0 0.325 0 0.224 1 0.251
1 0.186 1 0.224 2 0.2
2 0.163 2 0.184 3 0.2
3 0.163 3 0.184 4 0.2
4 0.163 4 0.184 0 0.149
Table 1: Ranking according to the Pagerank centrality. Nodes of the graph in Figure 2 are sorted according to the Pagerank. As for the eigencentrality case of Figure 2, the relative ordering of nodes can be changes modifying the value of the bias parameter .

Tran et al, in [tran2019dir_hypergraph_pagerank], use a similar approach fixing the bias parameter . However, they use a symmetrized version of the Laplacian instead of computing the dominant eigenvector from the transition matrix.

Tudisco et al. [tudisco2021eigencentrality] introduce a different approach to calculate the eigencentrality of a hyper-graph inspired by the HITS algorithm. Here, the importance of nodes and edges depends on the importance of the neighboring edges and nodes respectively (as opposed to using hub and authority centralities for nodes only).

The simple linear case translates, for the node ranking, to finding the dominant right eigenvector of the interaction matrix:

(8)

Unfortunately the interaction matrix is not always a function of the dynamics. However, if the interaction matrix is symmetric, as for the simple graph, the steady state assumes the simple form (we omit the dependence on ):

(9)

where represents the norm or the sum of all entries. In fact, one can see that the Eq. 9 defines a stationary state:

(10)

The elements of the interaction matrix are then proportional to the probability matrix :

(11)

and then can be derived by the dynamics except for a multiplicative factor .

iii.2 Community detection

Extending community detection algorithms that involve a measure of the dynamics is also at reach.

Consider modularity [newman2004finding] which is the objective function of a number of algorithms[blondel2008modularity, traag_leiden_2019]. In this framework the algorithm tries to maximize the modularity function which, in a symmetric graph, is:

(12)

where is the adjacency matrix, is the node degree (or strength in the weighted case), is the number of edges and is the community assigned to node .

While the Eq. 12 is a function of the linking pattern, it has been shown that it can be rewritten as a sum of auto-covariance of a random walk [Delvenne2010stability, Schaub2019multiscale]:

providing a dynamical interpretation of modularity maximization. In a symmetric graph, the above equation translates to:

(13)

that depends only on .

The above relation between modularity and auto-covariance maximization holds for the symmetric case where the probability matrix

is proportional to the adjacency matrix of the graph and the steady state probability distribution is proportional to the node degrees. Leicht and Newman extend the modularity function to directed networks 

[Leicht2008CommunitySI] disentangling the modularity definition from its dynamical interpretation. Inspired by the relation between covariance and modularity, Delvenne et al [Delvenne2010stability] provide an extension of Eq. III.2 to directed graphs and to different time scales. This formulation, which utilizes the dynamical system to cluster the graph nodes, let us extend the community detection algorithm to hyper-graphs [carletti2020stability].

Figure 3: Communities of the French-speaking twitter discussing vaccines during the Covid-19 pandemic. Fraction of outgoing hyper-edges from communities detected by modularity optimization when favouring tweets (; left plot) or retweets (; right plot), when considering diffusion of vaccine-critic information. The shift of prolific vaccine-critic communities of the upper part of the plots toward the right, when , shows that information reaching different communities comes mostly from highly retweeted tweets.

An online social network like Twitter, where the discussion follows a one-to-many pattern, represents a system where hyper-graphs are best suited. Figure 3 shows the community structure of users on the French-speaking part of Twitter, discussing vaccine related topics in the context of the COVID-19 pandemic (dataset from [faccin2022covid]). Each tweeting user represents the tail of a hyper-edge whose head nodes are the retweeting users, and communities are detected through the maximization of the auto-covariance. Here we plot for different values of the average visiting probability (the probability of visiting a node of a given partition, on average) and the escape probability (the probability of reaching a node on a different community) of each community. When comparing the communities computed with (each tweet has the same weight) or (each tweet weight is proportional to the number of its retweets) one can notice that the in the upper part of the plots (the most prolific communities), there is a shift toward higher outgoing flows. The dynamics corresponding to higher values of favors, for those communities, hyper-edges with larger heads (higher number of retweets).

The shift in Figure 3 uncover a higher probability of tweets/hyper-edges with larger cascades/heads to reach other communities. Figure 4 shows the escape probability of the larger communities. Notice that the probability of reaching nodes on different communities is an increasing function of in the range of values analyzed. The communities are recomputed independently at each value of and the same name is assigned if they overlap more than 90%. This account for some of the fluctuations due to merging and splitting of communities.

Figure 4: Escape probability per community. Note that most of the communities increases their capacity to reach other communities as the value of increase. Fluctuations are often due the splitting or merging of communities as changes. The filled area around the lines is proportional to the number of nodes in each community. For readability only communities with more than 5k users are reported.

We analyze the capacity of tweets of different cascade size to reach a different community of the original poster. Figure 5 supports this hypothesis showing the ratio of retweets reaching other communities per tweet class. Here one can notice the increasing probability of being retweeted outside of one’s community when the tweet become more and more popular.

Figure 5: Tweet capacity to reach outer communities. Proportion of retweets from other communities as a function of the retweet-cascade size. Results are robusts upon changes of the community structure as consequence of

Other algorithms.

Within the vast ecosystems of community (or other mesoscopic structures) detection algorithms, those which objective function only depends on the dynamics onf the system, can be extended as in this work. Some notable examples could be the map-equation [Rosvall_2009] which was extended to symmetric hyper-graphs in [carletti2020stability]. Another algorithm based on information theoretical arguments amenable to extension is the autoinformation state aggregation [faccin2021aisa].

Iv Discussion

We discussed how the analysis of higher order graphs can be performed through the analysis of the dynamical system coupled to the hyper-edge topology. When a transition matrix can be computed for the steady state of the dynamics, all measures that are function of the latter can be applied directly. One can assume that a function of the dynamics have the same results if dynamics on a hyper-graph and on the corresponding graph with identical transition matrix.

Prudence is necessary. Although an effective transition matrix may exist in some cases, to such transition matrix correspond a whole family of possible adjacency matrices which produce the same transition probabilities. While in the symmetric case one can naïvely analyze the interaction matrix of Eq.1 as an adjacency matrix of an effective graph, this can lead to misconceptions. In particular one may be tempted to analyze the linking pattern of such effective graph incurring into an improper extension of the measure of interest. In the directed case, the analysis of the interaction matrix does not directly relate to the dynamical behavior of the original system.

Other popular approaches involve an expansion of the hyper-edge pattern, e.g. clique expansion which assumes a specific dynamical model that may not match the original system. If the interest of the researcher is instead on the linking pattern of the hyper-graph, one must redefine the measure of interest, as in [tudisco2021eigencentrality, Estrada2006SubgraphCA] where the authors define a new eigencentrality or clustering coefficient adapted to this framework.

A topic of interest for further studies would be how to determine the dynamical model to use on the hyper-graph structure, in particular, in the linear case, the best value of . In many cases, as in the preset work, the choice fall on the researcher, and they need to determine the value of or to analyze a range of values. Another way would be to use one of the many model selection approaches applied to the measure of interest, such as the elbow or the plateau criterions.

Another topic of interest for further studies is determining when this framework is applicable to non-linear dynamics such non-Markovian [salnikov2016] dynamics or quantum mechanical systems [biamonte2019classicalquantum, Rossi_2013]. In the former the dynamics depends on the symbols visited in the past (the dynamics is linear on the set of present and past symbols). In the latter the quantum system evolves on the nodes, and, while there is no steady state, one may consider temporal averages or open quantum systems (coupled with a bath).

Acknowledgements.
MF thanks Michael Schaub for the insightful discussions on the topic. Part of this work has benefited from the financial support of the Agence Nationale de la Recherche (projects TRACTRUST - ANR-20-COVI-0102).

References