Computer systems are widely deployed to manage the business in industry and government. Ensuring the proper functioning of these systems is critical to the execution of the business. For example, if a system is compromised, the security of the customer data cannot be guaranteed; if certain components of a system have failures, the services hosted in the system may be interrupted. Maintaining the proper functioning of computer systems is a challenging task. System experts have limited visibility into systems, as the tools they use often give a partial view of the complex systems. This motivates the recent trend of leveraging system monitoring logs to offer intelligence in system management.
Temporal graphs are key data structures used to fuse information from heterogeneous sources [36, 39]. In computer systems, system monitoring data are represented as temporal graphs. For example, in cybersecurity, system call (syscall) logs provide a comprehensive way to capture system activities . Unlike its alternatives (e.g., file access logs , firewall , and network monitoring ) which provide partial information and are application-specific, syscall logs cover all interactions among system entities (e.g., processes, files, sockets, and pipes) over time. In Figure 1(a), a syscall log contains a sequence of events each of which describes at which time what kind of interactions happened between which system entities. Note that this syscall log also forms an equivalent temporal graph.
While temporal graphs provide an intuitive way to visualize system behaviors, another important step is to query such data and leverage the results to better comprehend system status. The question is how! It is usually burdensome to formulate useful queries to search system-generated temporal graphs, as they are complex with many tedious low-level entities. Consider the following scenarios.
Example 1 (Cybersecurity)
To ensure the security of an enterprise system, a system expert wants to know if there exists any information stealthy activity in the human resource department over the weekend. A hypothetical activity could involve three steps: someone remotely accessed an HR desktop by ssh, compressed several files, and transferred them to a remote server. In order to find such activity, one can submit a query like Figure 1(b) consisting of three components: “sshd-login”, “compress-files”, and “send-to-remote-server”, and perform search over syscall logs like Figure 1(a). However, such query cannot retrieve any useful information since the low-level entities (e.g., processes and files) recorded in the syscall logs cannot be directly mapped to any high-level activity like “sshd-login” or “compress-files”. In order to locate all “sshd-login” activities, one has to know which processes or files are involved in “sshd-login” and in what order over time these low-level entities are involved in order to write a query. This becomes very time consuming.
Besides querying risky behaviors, the formulated behavior queries can also be applied on the real-time monitoring data for surveillance and policy compliance checking.
Example 2 (Datacenter monitoring)
State-of-the-art system monitoring tools generate large-scale monitoring data as temporal graphs , where nodes are system performance alerts, and edges indicate dependencies between alerts. While these alerts suggest low-level anomalies (e.g., CPU usage is too high on server A, or there are too many full table joins on server B), system experts desire high-level knowledge about system behaviors: do these alerts result from disk failure or abnormal database workload? To search such high-level system behaviors, we have to know how alerts trigger each other and their temporal order . It is a daunting task for system experts to manually formulate such queries.
In addition, such query formulation problems also exist in other complex systems in the big data era.
Example 3 (Urban computing)
Modern cities generate a diverse array of data from heterogeneous sources, such as traffic flow, reports of sickness in cities, reports of food production, and so on . Information inferred from these sources are fused into temporal graphs, where nodes are events detected from different sources (e.g., traffic jam, high sickness rate, and decrease in food production yield), edges indicate relationships between events (e.g., two events are geographically close), and timestamps record when such relationships are detected. Domain experts are interested in high-level knowledge in urban systems: are these unusual events caused by river or air pollution? To formulate queries and search such knowledge, domain experts have to understand detailed temporal dependency patterns between events, which is extremely difficult for them.
Querying high-level system behaviors significantly reduces the complexity of evaluating system status, but it is quite difficult to formulate useful system behavior queries, referred to as behavior queries in this paper, because of the big semantic gap between the high-level abnormal activities and the low-level footprints of such activities. To address this problem, one approach is to collect monitoring data of target behaviors (e.g., “sshd-login”), model the raw monitoring data by heterogeneous temporal graphs to , and use the full graphs to formulate queries. Unfortunately, the raw data can be large and noisy. To overcome this challenge, instead of using the full graphs, we identify the most discriminative patterns for target behaviors and treat them as queries. Such queries (e.g., a few edges) are easier to interpret and modify, and are robust to noise. A discriminative pattern should frequently occur in the target activities, and rarely exist in other activities. One of the discriminative subgraph patterns for “sshd-login” is shown in Figure 1(c), which includes a few nodes/edges and is more promising for querying “sshd-login” from syscall logs.
To this end, we formulate the behavior query construction problem as a discriminative temporal graph mining problem: Given a positive set and a negative set of temporal graphs, the goal is to find the temporal graph patterns with maximum discriminative score. It is difficult to extend the existing mining techniques [11, 30] to solve this problem, since they mainly focus on non-temporal graphs, not temporal graphs (detailed discussion in Section 7.1).
In this paper, we propose that addresses the challenges to discriminative temporal graph pattern mining.
We have to consider both topology and edge temporal order while searching temporal graph space. To avoid redundant search, do we need another complex canonical labeling method like [11, 31] for mining temporal graphs? In our study, we find the temporal information in graphs allows us to explore temporal graph space in a more efficient manner. In particular, we propose a pattern growth algorithm without any complex canonical labeling. It guarantees that all promising patterns are covered, and no redundant search.
Since temporal graph space is huge, a naive exhaustive search is slow even for small temporal graphs. To speed up search, we first identify general cases where we can conduct pruning. Then we propose algorithms to minimize the overhead for discovering the pruning opportunities: (1) By encoding temporal graphs into sequences, a light-weight algorithm based on subsequence tests is proposed to enable fast temporal subgraph tests; and (2) we compress residual graph sets into integers such that residual graph set equivalence tests are performed in constant time.
Our major contributions are as follows. First, motivated by the need of queries in system monitoring applications, we identify a challenging query formulation problem in complex temporal graphs. Second, we propose the idea of using discriminative subgraph pattern mining to automatically formulate behavior queries, significantly easing the query formulation. Third, we develop that leverages temporal information to enable fast pattern mining in temporal graphs. Experimental results on real data show that the behavior queries constructed by are effective for behavior analysis in cybersecurity applications, with high precision and recall
, better than a non-temporal graph pattern based approach whose precision and recall areand , respectively. For mining speed, is 6-32 times faster than baseline methods.
2 Problem Formulation
As described earlier, our goal is to mine skeletons for behavior queries from temporal graphs derived from system monitoring data. In the following, we focus on cybersecurity applications for the ease of presentation, but the proposed ideas and techniques can also be applied to applications in other domains.
Temporal graph. A temporal graph is represented by a tuple , where (1) is a node set; (2) is a set of directed edges that are totally ordered by their timestamps; (3) is a function that assigns labels to nodes ( is a set of node labels); and (4) is a set of possible timestamps, non-negative integers on edges.
In practice, the syscall data of a behavior instance is collected from a controlled environment, where only one target behavior is performed. In most cases, the syscall data of a target behavior forms a temporal graph of no more than a few thousand of nodes/edges.
We target temporal graphs with total edge order in this work. Our empirical results show that this model performs quite well in identifying basic security behaviors, as these behaviors are usually finished by one thread. It also possesses computation advantages, in comparison with more complex temporal graph models. For the cases of concurrent edges, we provide a discussion in Section 5.
Figure 3 shows three temporal graphs. Note that multi-edges are allowed in temporal graphs as shown in . For simplicity, we examine temporal graphs with only node labels and edge timestamps in this paper. Our algorithms also work for temporal graphs with edge labels.
Temporal graph pattern. A temporal graph pattern is a temporal graph, where .
Unlike general temporal graphs where timestamps could be arbitrary non-negative integers, timestamps in temporal graph patterns are aligned (from 1 to ) and only total edge order is kept. In the following discussion, we use upper case letters (such as ) to represent temporal graph data, and use lower case letters (such as ) to represent abstract temporal graph patterns.
As it is ineffective and inefficient to take the entire raw graph as a behavior query, it will be better to use its discriminative subgraphs to capture the footprint of a behavior.
Temporal subgraph. Given two temporal graphs and , if and only if there exist two injective functions and such that (1) node mapping: , ; (2) edge mapping: , ; and (3) edge order preserved: , .
is a match of denoted as , when and are bijective functions.
Figure 3 shows an example of a temporal subgraph where . In particular, the subgraph in formed by edges of timestamps , , and is a match of .
Pattern frequency. Given a set of temporal graphs and a temporal graph pattern , the frequency of with respect to is defined as
Moreover, we differentiate two types of connected graphs among temporal graphs.
T-connected temporal graph. A temporal graph is T-connected if , the edges whose timestamps are smaller than form a connected graph.
In Figure 3, and are T-connected temporal graphs while is not, because the graph formed by edges with timestamps smaller than is disconnected.
In this work, we focus on mining T-connected temporal graph patterns for the following reasons. First, in pattern growth, T-connected patterns remain connected, while non T-connected patterns might be disconnected during the growth process, resulting in formidable explosion of pattern search space. Second, any non T-connected temporal graph is formed by a set of T-connected temporal graphs. In practice, we can use a single T-connected pattern or a set of T-connected patterns that in all could be a non T-connected pattern to form a behavior query. In the rest of this paper, T-connected temporal graphs are referred to as connected temporal graphs without ambiguity. Next we define the discriminative temporal graph pattern mining problem.
Given a set of positive temporal graphs and a set of negative temporal graphs , the goal is to find the connected temporal graph patterns with maximum , where is a discriminative score function with partial (anti-)monotonicity: (1) when is fixed, is smaller, is larger; and (2) when is fixed, is larger, is larger.
covers many widely used score functions including G-test, information gain, and so on . In practice, one could pick a discriminative score function that satisfies partial (anti-)monotonicity and best fits his/her query formulation task. Note that for the ease of presentation, the discriminative score of a pattern is also denoted as in the following discussion.
Behavior query formulation pipeline. Figure 2 shows a pipeline of collecting syscall logs for behaviors, finding patterns, and using them to construct graph pattern queries for searching behaviors from syscall data (temporal graphs) and retrieving interesting security knowledge. We take the behavior of sshd-login as an example.
The first step is to form input data. Relatively clean syscall logs for sshd-login are crawled from a closed environment, where sshd-login is independently run multiple times. Additionally, we also collect syscall logs where sshd-login is not performed and treat them as background syscall logs. The input of mining sshd-login behavior patterns is formed as follows: (1) the raw syscall logs of sshd-login are treated as a set of positive temporal graphs and (2) the raw background syscall logs are treated as a set of negative temporal graphs . In practice, we can also use the syscall logs for normal or abnormal sshd-login (e.g., intrusion) as positive datasets, which will generate graph pattern queries for normal and abnormal behaviors.
Given and , finds the most discriminative temporal graph patterns for sshd-login. To identify the patterns that best serve the purpose of behavior search, the patterns discovered by are further ranked by domain knowledge, including semantic/security implication on node labels and node label popularity among monitoring data. Top ranked patterns are then selected as queries to search sshd-login activities from a repository of syscall log data and see if there are abnormal/suspicious activities, e.g., too many times of sshd-login over a Saturday night.
overview. Our mining algorithm includes two key components: pattern growth and pattern space pruning.
Pattern growth guides the search in pattern space. It conducts depth-first search: starting with an empty pattern, growing it into a one-edge pattern, and exploring all possible patterns in its branch. When one branch is completely searched, the algorithm continues to explore the branches initiated by other one-edge patterns. The key challenges in this component is how to avoid repeated pattern search and how to cover the whole pattern space.
Pattern space pruning is a crucial step to speed mining processes. The underlying pattern space could be large, and a naive search algorithm cannot scale. Therefore, effective pruning algorithms are desired. The key problems in this component is how to identify general pruning opportunities and how to minimize the overhead in pruning.
3 Temporal Graph Growth
In this section, we discuss the pattern growth algorithm in . In particular, we demonstrate (1) how temporal information in graphs enables efficient pattern growth without repetition, and (2) the general principles to grow temporal graph patterns so that all possible connected temporal graph patterns will be covered.
3.1 Growth without Repetition
Pattern growth is more efficient in temporal graphs, compared with its counterpart in non-temporal graphs.
It is costly to conduct pattern growth for non-temporal graphs. To grow a non-temporal pattern to a specific larger one, there exist a combinatorial number of ways. In order to avoid repeated computation, we need extra computation to confirm whether one pattern is a new pattern or is a discovered one. This results in high computation cost, as graph isomorphism is inevitably involved. To reduce the overhead, various canonical labeling techniques along with their sophisticated pattern growth algorithms [8, 11, 31] have been proposed, but the cost is still very high because of the intrinsic complexity in graph isomorphism.
Unlike non-temporal graphs, we can develop efficient algorithms for temporal graph pattern growth. First, we can decide whether two temporal graph patterns are identical in linear time. Second, these is at most one possible way to grow a temporal graph pattern to a specific larger one. Next, we show why these properties stand.
The computation advantages of temporal graphs originate from the following property.
Let and be temporal graph patterns. If , the mappings and between them are unique.
The proof is detailed in Appendix A.
With Lemma 1, we further prove the efficiency of determining whether two temporal graph patterns are matched.
If and are temporal graph patterns, then can be determined in linear time.
The proof is detailed in Appendix B.
Pattern growth for temporal graphs will be more efficient, when it is guided by consecutive growth.
Consecutive growth. Given a connected temporal graph pattern of edge set and an edge , adding into is consecutive growth, if (1) it results in another connected temporal graph pattern; and (2) .
Figure 4 demonstrates how grows to by consecutive growth. Consecutive growth guarantees a connected temporal graph pattern will form another connected temporal graph pattern without repetition.
Let and be connected temporal graph patterns with . If pattern growth is guided by consecutive growth, then (1) either there exists a unique way to grow into , (2) or there is no way to grow into .
The proof is detailed in Appendix C.
3.2 Growth Options
To guarantee the quality of discovered patterns, we need to ensure our search algorithm can cover the whole pattern space. In this work, we identify three growth options to achieve the completeness. Let be a connected temporal graph pattern with node set . We can grow by consecutive growth as follows.
Forward growth: growing by an edge is a forward growth if and .
Backward growth: growing by an edge is a backward growth if and .
Inward growth: growing by an edge is an inward growth if and .
Figure 5 illustrates the three growth options. Note that inward growth allows multi-edges between node pairs.
The three growth options provide a guidance to conduct a complete search over pattern space.
Let be a search algorithm following consecutive growth with forward, backward, and inward growth. Algorithm guarantees (1) a complete search over pattern space, and (2) no pattern will be searched more than once.
The proof is detailed in Appendix D.
4 Pruning Temporal Graph Space
In this section, we investigate how to prune search space by the unique properties in temporal graphs. First, we explore the general cases where we can prune unpromising branches. Next, we identify the major overhead for discovering pruning opportunities, and leverage the temporal information in graphs to minimize the overhead.
4.1 Naive Pruning Conditions
A straightforward pruning condition is to consider the upper bound of a pattern’s discriminative score. Given a temporal graph pattern , the upper bound of indicates the largest possible discriminative score that could be achieved by ’s supergraphs. Let and be positive graph set and negative graph set, respectively. Since , and , we can derive the following upper bound,
This upper bound is theoretically tight; however, it is ineffective for pruning in practice . In the rest of this section, we discuss general pruning opportunities inspired by temporal sub-relations.
4.2 Pruning by Temporal Sub-relations
For a temporal graph pattern, we denote the graph data that need to be considered for growing this pattern as a set of residual graphs.
Let be a subgraph of . If we remove the edges in whose timestamps are no larger than the largest edge timestamp in , then we form a residual graph.
Residual graph. Given a temporal graph and its subgraph , is ’s residual graph with respect to , where (1) satisfies , ; and (2) is the set of nodes that are associated with edges in . The size of is defined as (i.e., the number of edges in ).
Given a residual graph , its residual node label set is defined as .
Figure 6 demonstrates an example of residual graph, where is a subgraph of , is ’s residual graph with respect to , and is its residual node set.
Let be a set including all the subgraphs in that match a temporal graph pattern . Given and , we define positive residual graph set as
Given , its residual node label set is defined as
Similarly, we can define negative residual graph set and its residual node label set .
Next, we introduce another important property that helps us identify pruning opportunities.
Given a temporal graph set and two temporal graph patterns , if , then the node mapping between and is unique.
We sketch the proof as follows. First, we prove if the node mapping between and is not unique, any match for includes multiple matches for . Then we prove if any match for includes multiple matches for , will never be true. The detailed proof is stated in Appendix E.
In the following, we present subgraph pruning and supergraph pruning. In the following discussion, for a temporal graph pattern , we use ’s branch to refer to the space of patterns that are grown from , and use to denote the largest discriminative score discovered so far.
Let and be temporal graph patterns where is discovered before . If (1) is a temporal subgraph of ; (2) they share identical positive residual graph sets; and (3) for those nodes in that cannot match to any nodes in , their labels never appear in ’s residual node label set, then we can conduct subgraph pruning on .
Subgraph pruning. Given a discovered pattern and a pattern of node set , if (1) , (2) , and (3) (where and is the set of nodes that map to nodes in ), we can prune the search on ’s branch, if the largest discriminative score for patterns in ’s branch is smaller than .
Figure 7 illustrates the idea in subgraph pruning. In the mining process, we reach a pattern , and we also notice that there exists a discovered pattern , which satisfies the conditions in subgraph pruning. Therefore, pattern growth in ’s branch suggests how to grow to larger patterns (e.g., growing to indicates we can grow to ). Since none of the patterns in ’s branch have the score , the patterns in ’s branch cannot be the most discriminative ones as well, which can be safely pruned.
Subgraph pruning prunes pattern space without missing any of the most discriminative patterns.
The proof is detailed in Appendix F.
Similar to subgraph pruning, we can perform supergraph pruning. Let and be temporal graph patterns where is discovered before . If (1) is a temporal subgraph of , (2) they share identical positive and negative residual graph sets, and (3) they have the same number of nodes, we can conduct supergraph pruning on .
Supergraph pruning. Given two patterns and , where is discovered before and is not grown from , if (1) , (2) , (3) , and (4) and have the same number of nodes, the search in ’s branch can be safely pruned, if the largest discriminative score for ’s branch is smaller than .
Figure 8 demonstrates the idea in supergraph pruning. In the mining process, a temporal graph pattern is reached, and there is another pattern discovered before , which satisfies the conditions in supergraph pruning. Therefore, the growth knowledge in ’s branch suggests how to grow to larger patterns. Since none of the patterns in ’s branch are the most discriminative, we can infer the patterns in ’s branch are unpromising as well, and the search in ’s branch can be safely pruned.
Supergraph pruning prunes pattern space without missing the most discriminative patterns.
The proof is detailed in Appendix G.
Performing subgraph pruning and supergraph pruning guarantees the most discriminative patterns will still be preserved.
Theorem 2 identifies general cases where we can conduct pruning in temporal graph space; however, it works only if the overhead for discovering these pruning opportunities is small enough. The major overhead of subgraph pruning and supergraph pruning comes from two sources: (1) temporal subgraph tests (e.g., ), and (2) residual graph set equivalence tests (e.g., ). For example, to mine patterns for “sshd-login” behavior, the mining process involves more than 70M temporal subgraph tests and 400M residual graph set equivalence tests. These overhead is not negligible, and it significantly degrades the efficiency of the pruning algorithm. Next, we investigate how to minimize these overhead.
4.3 Temporal Subgraph Test
In this section, we discuss how to leverage the temporal information in graphs to minimize the overhead from temporal subgraph tests. First, we propose an encoding scheme that represent temporal graphs by sequences. Second, we develop a light-weight algorithm based on subsequence tests.
Similar to subgraph test for non-temporal graphs, it is difficult to efficiently perform temporal subgraph tests.
Given two temporal graphs and , it is NP-complete to decide .
We sketch the proof as follows. First, we prove the NP-hardness by reducing clique problem  to temporal subgraph test problem. By transforming non-temporal graphs into temporal graphs in polynomial time, we show we solve clique problem by temporal subgraph tests, which implies temporal subgraph test problem is at least as hard as clique problem. Second, we prove temporal subgraph test problem is NP by showing we can verify its solution in polynomial time. The proof is detailed in Appendix H.
Since edges are totally ordered in temporal graphs, it is possible to encode temporal graphs into sequences.
After temporal graphs are represented as sequences, it is possible to enable faster temporal subgraph tests using efficient subsequence tests.
Based on these insights, we propose a light-weight temporal subgraph test algorithm. In particular, this algorithm consists of two components: (1) a sequence-based temporal graph representation, and (2) a temporal subgraph test algorithm based on subsequence tests.
Before we dive into the technical details, we review the definition of subsequence. Let and be two sequences. If there exist such that , , is a subsequence of , denoted as .
Sequence-based representation. A temporal graph pattern can be represented by two sequences.
Node sequence is a sequence of labeled nodes. Given is traversed by its edge temporal order, nodes in are ordered by their first visited time. Any node of appears only once in .
Edge sequence is a sequence of edges in , where edges are ordered by their timestamps;
Figure 9 illustrates examples of sequence-based representation. In and , node labels are represented by letters, and nodes of the same labels are differentiated by their node IDs represented by integers in brackets. Node labels in are associated with node IDs as subscripts. Note that when we compare node labels, their subscripts will be ignored (i.e., ). Each edge in is represented by the following format , where is the source node ID and is the destination node ID.
Given two temporal graphs and , if , we expect and . However, when , may not be true. As shown in Figure 9, because the first visited time of the node with label is inconsistent in and .
To address this issue, we propose enhanced node sequence . Let be a temporal graph. is a sequence of labeled nodes in . Given is traversed by its edge temporal order, is constructed by processing each edge as follows. (1) If is the last added node in the current , or is the source node of the last processed edge, will be skipped; otherwise, will be added into . (2) Node will be always added into . Note that nodes in might appear multiple times in . Figure 9 shows the enhanced node sequences of and .
With the support from enhanced node sequence, we are ready to build the connection between temporal subgraph tests and subsequence tests.
Two temporal graphs if and only if
, where the underlying match forms an injective node mapping from nodes in to nodes in ;
, where is an edge sequence where the nodes in are replaced by the nodes in via the node mapping .
The proof is detailed in Appendix I.
As shown in Figure 9, and are two temporal graphs satisfying . The node sequence of is a subsequence of the enhanced node sequence of with the injective node mapping , , , and . Therefore, we obtain so that .
Subsequence-test based algorithm. A temporal subgraph test algorithm is derived from Lemma 5. Given temporal graphs and , the algorithm performs as follows.
Search an injective node mapping between and such that ;
Test whether . If yes, the algorithm terminates and returns ; otherwise, the algorithm searches next qualified node mapping. If such a node mapping exists, repeat step 1 and 2; otherwise, the algorithm terminates with .
Although we can perform subsequence tests in linear time, many possible node mappings may exist. Among all the possible mappings, a large amount of them are false mappings, which are not injective. To further improve the speed, we adapt existing pruning techniques for subsequence matching to temporal subgraph tests. The idea is to identify false mappings as early as possible by leveraging node labels, local neighborhood information, and processed prefixes as conditions to prune unpromising search branches. We detail these pruning techniques in Appendix J.
4.4 Residual Graph Set Equivalence
In this section, we discuss how to efficiently test equivalent residual graph sets by leveraging temporal information in graphs. A naive approach applies a linear scan algorithm. Since residual graph set equivalence tests are frequently employed by subgraph and supergraph pruning, repeated linear scans causes significant overhead and suffers poor efficiency.
Let and be temporal graph patterns. Consider and are the matches of and in , respectively. Since edges in temporal graphs are totally ordered, we can derive the following result: is equivalent to if and only if . Thus, we can efficiently conduct residual graph set equivalence tests as follows.
Given temporal graph patterns and with , and a set of graphs , if and only if , where
The proof is detailed in Appendix K. Remark. We only need to pre-compute once by a linear scan over , and the following residual graph set equivalence tests are performed in constant time.
5 Discussion: Concurrent Edges
In a system with parallelism and concurrency, its monitoring data may generate concurrent edges (i.e., edges sharing identical timestamps). In the following, we discuss how our technique can handle such cases.
First, we can extend to mine patterns of concurrent edges by the following modifications.
For temporal graph representation, instead of using a sequence of edges, we use a sequence of concurrent subgraphs, each of which includes all the edges sharing identical timestamps.
In terms of pattern growth, if the added edge has a larger timestamp compared with the last added edge, follow the growth algorithm in ; if the added edge share the same timestamp with the last added edge, follow the growth algorithm in ; and we never add edges with smaller timestamps. To avoid repeated patterns, each concurrent subgraph is encoded by canonical labeling.
For subgraph tests in pruning, we need to replace node matching based on labels with concurrent subgraph matching based on subgraph isomorphism, and ensure node mappings are injective.
Note that the computation complexity in the extended will be increased, as the costly subgraph isomorphism is unavoidable for dealing with the non-temporal graphs formed by concurrent edges.
Second, instead of modifying the mining algorithm, we can transform concurrent edges into total-ordered edges. Despite of the existence of concurrency, data collectors can sequentialize concurrent events based on pre-defined policies [28, 39] (e.g., randomly assigning a total time order for concurrent edges). In this way, we use the monitoring data with an artificial total order to approximate the original data, and apply without modification. When there are a small portion of concurrent edges with minor accuracy loss, this method benefits the efficiency of .
6 Experimental Results
In this section, we evaluate the proposed algorithms for behavior query discovery using real system activity data (i.e., syscall logs). In particular, we focus on two aspects: (1) the effectiveness of the behavior queries found by our algorithms, and (2) the efficiency of the proposed algorithms.
We start with the description for the datasets investigated in our experimental study.
|Behavior||Avg. nodes||Avg. edges||Total labels||Size|
Training data. We collect 13 datasets with in total temporal graphs to mine behavior queries for 12 different behaviors. In total, it contains nodes and edges derived from the syscall logs including behaviors of interest and background system activities.
We focus on 12 behaviors as representatives for the basic behaviors that have drawn attention in cybersecurity study [2, 3, 10, 33]. For each behavior, we collect 100 temporal graphs based on the syscall logs generated from 100 independent executions of the behavior, as we find the sample size of 100 already achieves good precision (97) and recall (91). For background system activities, temporal graphs are sampled from days’ syscall logs generated by a server without performing any target behaviors.
The statistics of the training data are shown in Table 1. Although the size of a single temporal graph is not large, we find discriminative patterns within the data can involve up to 45 edges, which is a large number for pattern mining problems because of the exponential number of sub-patterns.
To evaluate how the effectiveness and efficiency are affected by different amounts of training data, we vary the amount of used training data from to , where means 1 of training data are used for behavior query discovery, and means we consider all the data.
In addition, we generate synthetic datasets to evaluate the scalability of . The synthetic datasets are created based on the training data: we generate datasets -2, -4, -6, -8, and -10 by replicating each graph in the training data 2, 4, 6, 8, and 10 times, respectively.
Test data. Test data are used to evaluate the accuracy of the behavior queries found by our algorithms. They are obtained from an independent data collection process: we collect another seven days’ syscall log data, which forms a large temporal graph with nodes and edges. The test data contain behavior instances of the 12 target behaviors. Note that our focus in this paper is query formulation instead of pattern query processing. Based on the patterns discovered from training data, we formulate behavior queries, and search their existence from the test data by existing techniques .
More details about training/test data collection are presented in Appendix L.
Implementation. For effectiveness, we consider and as baselines. (1) employs non-temporal graph patterns for behavior query discovery. In particular, we remove all the temporal information in the training data, apply existing algorithms  to mine discriminative non-temporal graph patterns for each behavior, and use the discovered patterns to formulate non-temporal behavior queries. (2) searches behavior instances by keyword queries using a set of discriminative node labels. The discriminativeness of a node label is measured by the same score function for temporal graph patterns. Top- discriminative node labels are selected for a query of node labels. A match of a query is a set of nodes, where its node label set is identical to the node label set specified in the query, and its spanned time interval is no longer than the longest observed lifetime of the target behavior.
For efficiency, we implement five baseline algorithms to demonstrate the contributions of each component in . All the baselines apply the proposed pattern growth algorithm and the naive pruning condition stated in Section 4.1. (1) employs the pruning condition in Lemma 4. (2) considers the pruning condition in Proposition 2. (3) uses all the pruning conditions but applies a graph index based algorithm for temporal subgraph tests. In particular, we index one-edge substructures, and use efficient algorithms to join partial matches into full matches . (4) considers all the pruning conditions but performs temporal subgraph tests by a modified VF2 algorithm . (5) uses all the pruning conditions but performs residual graph set equivalence tests via a linear scan algorithm.
In addition, we employ information gain, G-test , as well as the function ( is set to ) adopted in  as discriminative score functions. In our experiment, we find these score functions deliver a common set of discriminative patterns.
All the algorithms are implemented in C++ with GCC 4.8.2, and all the experiments are performed on a server with Ubuntu 14.04, powered by an Intel Core i7-2620M 2.7GHz CPU and 32GB of RAM. Each experiment is repeated times, and their average results are presented.
6.2 Effectiveness of Behavior Queries
We evaluate the effectiveness of from three aspects. (1) For different behaviors, how accurately can behavior queries suggested by search behavior instances from system activity data? (2) How does query accuracy vary when pattern size in queries changes? (3) How does the amount of training data affect query accuracy?
Precision and recall are used as the metrics to evaluate the accuracy. Given a target behavior and its behavior query, a match of this behavior query is called an identified instance. An identified instance is correct, if the time interval during which the match happened is fully contained in a time interval during which one of the true behavior instances was under execution. A behavior instance is discovered, if the behavior query can return at least one correct identified instance with respect to this behavior instance. The precision and recall of a behavior query are defined as follows.
Note that when returns multiple discriminative patterns that have the same highest discriminative score, the returned patterns are further ranked by a score function based on domain knowledge. From all the discriminative patterns, top- patterns are used to build behavior queries. The details about the domain knowledge based score function is discussed in Appendix M.
Figure 10 illustrates a few discovered discriminative patterns. For sshd-login behavior, the unique interaction pattern among system entities makes it accurate for search purpose. Note that this pattern does not include any node with “sshd” in the label. This indicates that keyword-based techniques that simply use application names as keywords (e.g., sshd) cannot find such highly accurate patterns. For file download behaviors, it is the distinct patterns of how to access libraries and sockets that differentiate wget-based download from ftp-based download.
|Metric||Precision ()||Recall ()|
Table 2 shows the precision and recall of behavior queries on all the 12 behaviors. The size (i.e., the number of edges) of the behavior queries suggested by and is fixed as , using all the training data. employs the top- discriminative node labels to query each behavior. First, the behavior queries suggested by accurately discover behavior instances. Over all the behaviors, its average precision and recall are and , respectively. Second, the queries provided by only achieve of precision, suffering significantly higher false positive rate. This result indicates the importance of temporal information in searching system behaviors. Third, the queries suggested by consistently outperform the queries formed by . These results confirm that temporal graph patterns discovered by provide high-quality skeletons to formulate accurate behavior queries.
Figure 11 demonstrates how the precision and recall of behavior queries suggested by vary while their query size ranges from to . All the training data were used in this experiment, and the average precision and recall over all the behaviors are reported. First, when the query size increases, the precision increases, but the recall decreases. In general, increasing behavior query size improves query precision at the cost of a slightly higher false negative rate. Second, when the query size goes beyond , we observe little improvement on precision or loss on recall.
Figure 12 suggests how precision and recall of behavior queries vary as the amount of used training data is changed from to . Note that behavior query size is fixed as , and the average precision and recall over all the behaviors are presented. First, more training data bring higher precision and recall. For example, when the amount of used training data increases from to , the precision of the constructed behavior queries increase from to . Second, when the amount of used training data increases, we observe a diminishing return for both precision and recall.
6.3 Efficiency in Behavior Query Discovery
The efficiency of is evaluated from two aspects. (1) How is the efficiency of compared with baseline algorithms? (2) How is the efficiency of affected by different amounts of used training data? Note that behavior query discovery is an offline step: we only need to mine patterns once, and then use the patterns to formulate queries serving online search demands.
Response time is used as the metric to evaluate the efficiency. Given input temporal graph sets, the response time of an algorithm is the amount of time the algorithm spends in mining all discriminative patterns.
Figure 13 demonstrates response time of all algorithms over small, medium, and large size behaviors as categorized in Table 1. All the training data are used in this experiment, and the discovered patterns have up to 45 edges.
First, consistently outperforms the baseline algorithms over all the target behaviors. is up to and times faster than and . cannot finish the mining tasks for medium and large behaviors within 2 days. We notice most of the pruning opportunities come from subgraph pruning, while supergraph pruning brings additional performance improvement.
Second, performs up to , , and times faster than , , and , respectively. has to frequently build graph indexes for each discovered patterns during the whole mining process, which involves high overhead. Indeed, graph indexing is more suitable for querying large graphs, where we can build graph indexes offline. In our case, a light-weighted temporal subgraph test algorithm performs better. In sum, the performance improvement highlights the importance of minimizing the overhead in temporal subgraph tests and residual graph set equivalence tests.
Third, for small, medium, and large behaviors, can complete mining tasks within 6 seconds, 4 minutes, and 26 minutes, respectively. We also noticed that mines all discriminative patterns with no more than 6 edges within one minute for all the behaviors, while 6-edge patterns can achieve good search accuracy as shown before.
Figure 14 presents the response time of as the size of the largest patterns that are allowed to explore is set to , , , , and . When the size increases, the response time of increases. When the size is set as 5, can finish the mining tasks within seconds for all the behaviors,.
Empirical probabilities that pruning conditions are triggered on behaviors of different sizes
Table 3 shows the empirical probabilities that subgraph and supergraph pruning are triggered when is processing a pattern for behaviors of different sizes. The high trigger rate of subgraph pruning is consistent over all the behaviors, which explains its high pruning power.
Figure 15 shows how the response time of is affected by using different amounts of training data. Ranging from to , the response time of increases linearly when more training data is considered. The scalability test on synthetic datasets (detailed in Appendix N) also shows that the response time of linearly scales with the size of training data. From the training data with up to 20M nodes and 80M edges, can mine all discriminative patterns of up to 45 edges within 3 hours.
The experimental results are summarized as follows. First, behavior queries discovered by achieve high precision () and recall (), while the baseline algorithms and suffers poor accuracy. Second, consistently outperforms the baseline algorithms in terms of efficiency, and is up to , , and times faster than , , and , respectively.
7 Related Work
7.1 Discriminative Graph Pattern Mining
Discriminative graph pattern mining is one of the feature selection methods that is widely applied in a variety of graph classification tasks[21, 25, 30].
In general, two directions have been investigated to speed up mining discriminative non-temporal graph patterns. One direction is to find discriminative patterns early such that unpromising search branches can be pruned early . The other direction relies on approximate search that finds good-enough graph patterns . In addition, a few studies [19, 20, 27] focus on mining graph patterns from a sequence of graph snapshots.
It is difficult to extend existing work on non-temporal graphs to mine temporal graph patterns. The key problem is how to deal with timestamps in the mining process.
One possibility is to ignore timestamps: mine discriminative non-temporal patterns by existing approaches, and then use timestamps to find discriminative temporal patterns from the non-temporal patterns. This method has two drawbacks. First, since canonical labeling on non-temporal graphs [11, 31] have difficulties in dealing with multi-edges, we have to collapse multi-edges into a single edge. In this way, the final result will be partial, as it excludes patterns with multi-edges. Second, a large number of temporal patterns may share the same non-temporal patterns, and a discriminative non-temporal pattern may result in no discriminative temporal pattern. The redundancy in non-temporal patterns will bring potential scalability problems.
Another possibility is to consider timestamps as labels. The temporal patterns discussed in this paper focus on the temporal order instead of exact timestamps. Therefore, it is difficult to mine the desired patterns by the approaches for non-temporal patterns.
Our work is different from existing works. First, unlike existing studies that mainly deal with non-temporal graphs, we propose a solution to temporal graphs. Second, compared with existing works on graph snapshots, we study more flexible temporal graph patterns. The graph patterns defined in [19, 20, 27] are too rigid to support applications in system management.
7.2 Temporal Graph Management
Recent studies on temporal graphs strive to develop cost-effective solutions to management problems.
, and anomaly detection.
Graph indexing, compression, and partitioning have been proposed to accelerate query processing for subgraph matching , reachability , community searching , and neighborhood aggregation . Incremental computation has been investigated for queries including shortest-path , SimRank , and cluster searching . In addition, Gao et al.  exploited distributed computation to monitor subgraph matching queries.
Data storage is a critical component in temporal graph management. Grace  examined graph update transactions that incorporate dynamic changes into the latest graph snapshot. Chronos  investigated temporal data locality for query processing. Efficient ways to store and retrieve graph snapshots have been studied in [12, 14].
Unlike these works, our focus is to develop cost-effective algorithms that help users formulate meaningful temporal graph queries, bridging the gap between users’ knowledge and temporal graph data.
Computer system monitoring generates huge temporal graphs that record the interaction of system entities. While these graphs are promising for system experts to query behaviors for system management, it is also difficult for them to compose queries as it involves many tedious low-level system entities. We formulated this query formulation problem as a discriminative temporal graph mining problem, and introduced to mine discriminative patterns that can be taken as query templates for building more complex queries. leverages temporal information in graphs to enable efficient pattern space exploration and prune unpromising search branches. Experimental results on real system data show that is 6-32 times faster than baseline methods. Moreover, the discovered patterns were verified by system experts: they achieved high precision () and recall ().
Acknowledgement. We would like to thank the anonymous reviewers for the helpful comments on earlier versions of the paper. This research was partially supported by the Army Research Laboratory under cooperative agreements W911NF-09-2-0053 (NS-CTA), NSF IIS-1219254, and NSF IIS-0954125. The views and conclusions contained herein are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notice herein.
-  Splunk. http://www.splunk.com/.
-  Ssh brute force - the 10 year old attack that still persists. http://blog.sucuri.net/2013/07/.
-  U. Bayer, I. Habibi, D. Balzarotti, E. Kirda, and C. Kruegel. A view on current malware behaviors. In LEET, 2009.
-  W. R. Cheswick, S. M. Bellovin, and A. D. Rubin. Firewalls and Internet Security: Repelling the Wily Hacker. 2003.
-  L. P. Cordella, P. Foggia, C. Sansone, and M. Vento. A (sub) graph isomorphism algorithm for matching large graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004.
-  J. Gao, C. Zhou, J. Zhou, and J. X. Yu. Continuous pattern detection over billion-edge graph using distributed framework. In ICDE, 2014.
-  W. Hant, Y. Miao, K. Li, M. Wu, F. Yang, L. Zhou, V. Prabhakaran, W. Chen, and E. Chen. Chronos: a graph engine for temporal graph analysis. In EuroSys, 2014.
-  J. Huan, W. Wang, and J. Prins. Efficient mining of frequent subgraphs in the presence of isomorphism. In ICDM, 2003.
-  X. Huang, H. Cheng, L. Qin, W. Tian, and J. X. Yu. Querying k-truss community in large and dynamic graphs. In SIGMOD, 2014.
-  L. Invernizzi, S.-J. Lee, S. Miskovic, M. Mellia, R. Torres, C. Kruegel, S. Saha, and G. Vigna. Nazca: Detecting malware distribution in large-scale networks. In NDSS, 2014.
N. Jin, C. Young, and W. Wang.
Gaia: graph classification using evolutionary computation.In SIGMOD, 2010.
-  U. Khurana and A. Deshpande. Efficient snapshot retrieval over historical graph data. In ICDE, 2013.
-  S. T. King, Z. M. Mao, D. G. Lucchetti, and P. M. Chen. Enriching intrusion alerts through multi-host causality. In NDSS, 2005.
-  A. G. Labouseur, P. W. Olsen, and J.-H. Hwang. Scalable and robust management of dynamic graph data. In BD3 at VLDB, 2013.
-  P. Lee, L. V. Lakshmanan, and E. E. Milios. Incremental cluster evolution tracking from highly dynamic network data. In ICDE, 2014.
-  R. G. Michael and S. J. David. Computers and intractability: a guide to the theory of np-completeness. 1979.
-  J. Mondal and A. Deshpande. Eagr: Supporting continuous ego-centric aggregate queries over large dynamic graphs. In SIGMOD, 2014.
-  V. Prabhakaran, M. Wu, X. Weng, F. McSherry, L. Zhou, and M. Haradasan. Managing large graphs on multi-cores with graph awareness. In USENIX ATC, 2012.
-  G. Qin, L. Gao, J. Yang, and J. Li. Evolution pattern discovery in dynamic networks. In ICSPCC, 2011.
-  S. Ranu, M. Hoang, and A. Singh. Mining discriminative subgraphs from global-state networks. In KDD, 2013.
-  S. Ranu and A. K. Singh. Graphsig: A scalable approach to mining significant subgraphs in large graph databases. In ICDE, 2009.
-  C. Ren, E. Lo, B. Kao, X. Zhu, and R. Cheng. On querying historical evolving graph sequences. In VLDB, 2011.
-  K. Sricharan and K. Das. Localizing anomalous changes in time-evolving graphs. In SIGMOD, 2014.
-  Z. Sun, H. Wang, H. Wang, B. Shao, and J. Li. Efficient subgraph matching on billion node graphs. In VLDB, 2012.
-  M. Thoma, H. Cheng, A. Gretton, J. Han, H.-P. Kriegel, A. J. Smola, L. Song, S. Y. Philip, X. Yan, and K. M. Borgwardt. Near-optimal supervised feature selection among frequent subgraphs. In SDM, 2009.
-  W. Venema. TCP wrapper: Network monitoring, access control, and booby traps. In USENIX Security, 1992.
-  B. Wackersreuther, P. Wackersreuther, A. Oswald, C. Böhm, and K. M. Borgwardt. Frequent subgraph discovery in dynamic networks. In MLG, 2010.
-  P. Wang, H. Wang, M. Liu, and W. Wang. An algorithmic approach to event summarization. In SIGMOD, 2010.
-  H. Wu, J. Cheng, S. Huang, Y. Ke, Y. Lu, and Y. Xu. Path problems in temporal graphs. In VLDB, 2014.
-  X. Yan, H. Cheng, J. Han, and P. S. Yu. Mining significant graph patterns by leap search. In SIGMOD, 2008.
-  X. Yan and J. Han. gspan: Graph-based substructure pattern mining. In ICDM, 2002.
-  Y. Yang, H. Gao, J. X. Yu, and J. Li. Finding the cost-optimal path with time constraint over time-dependent graphs. In VLDB, 2014.
-  H. Yin, D. Song, M. Egele, C. Kruegel, and E. Kirda. Panorama: capturing system-wide information flow for malware detection and analysis. In CCS, 2007.
-  W. Yu, X. Lin, and W. Zhang. Fast incremental simrank on link-evolving graphs. In ICDE, 2014.
-  S. Zhang, J. Yang, and W. Jin. Sapper: subgraph indexing and approximate matching in large graphs. In VLDB, 2010.
-  Y. Zheng, H. Zhang, and Y. Yu. Detecting collective anomalies from multiple spatio-temporal datasets across different domains. In SIGSPATIAL, 2015.
-  A. D. Zhu, W. Lin, S. Wang, and X. Xiao. Reachability queries on large dynamic graphs: A total order approach. In SIGMOD, 2014.
-  B. Zong, R. Raghavendra, M. Srivatsa, X. Yan, A. K. Singh, and K.-W. Lee. Cloud service placement via subgraph matching. In ICDE, 2014.
-  B. Zong, Y. Wu, J. Song, A. K. Singh, H. Cam, J. Han, and X. Yan. Towards scalable critical alert mining. In KDD, 2014.
Appendix A Proof of Lemma 1
Without loss of generality, we assume and .
First, we prove is unique. Since and are temporal graph patterns, we have , and , . Because and , we can derive matches only if (in order to preserve total edge order). Thus, the uniqueness of is proved.
Second, we prove is unique. Since is unique, the edge mapping between and is unique, and then we can derive the node mapping is also unique.
Therefore, we have proved the uniqueness of and .
Appendix B Proof of Lemma 2
We first present the linear algorithm, and then prove its correctness. Suppose , , , and .
Algorithm. We conduct a linear scan over edges in . For each edge , we locate the edge . If such an edge exists, we verify if the mapping from to and the mapping from to are still one-to-one. If both are, we confirm matches . Then only if all the edges in find their matches in .
Correctness. We prove the correctness by showing the above algorithm finds two bijective functions and . First, since the linear scan follows the unique way to match edge timestamps between and and , is found and bijective. Second, the verification phase in the above algorithm guarantees the node mapping is one-to-one; moreover, is a full mapping because and all the nodes in and find their mapping in the verification phase. Therefore, we have proved the correctness of the linear algorithm.
Appendix C Proof of Lemma 3
Let and be connected temporal graph patterns with . Suppose the edge sets of and are and , respectively. Then we need to conduct steps of consecutive growth to grow into another pattern . If there exists , then we can tell it is possible to grow into ; otherwise, there is no way to grow to . Next we prove if we can grow into , then the steps of consecutive growth is unique.
Assume that (1) is a sequence of consecutive growth that grows into with , (2) is another sequence of consecutive growth that grows into with , and (3) is distinct from as cannot match . On the one hand, since and , we can infer by the bijective mapping functions. On the other hand, by the definition of consecutive growth, the linear scan algorithm from Lemma 2 will decide cannot match , since there exists at least one edge from that cannot match the edge in sharing the same timestamp, which contradicts with . Thus, is identical to , and the steps of consecutive growth is unique.
In sum, we have proved the correctness of Lemma 3.
Appendix D Proof of Theorem 1
We first justify the zero-redundancy in pattern search, and then prove the completeness.
Let be a connected temporal graph pattern. By Lemma 3, we can tell consecutive growth guarantees there exists a unique way to grow an empty pattern into . Thus, there is no way to search more than once.
For the completeness, we establish the proof by mathematical induction. Let be the number of edges in a temporal graph pattern.
Basis: we prove the completeness holds for . Since -edge patterns are formed by distinct edges, it is easy to see all -edge patterns will be covered.
Inductive step: we prove if the completeness holds for , then it holds for .
Assume the completeness holds for . Then we have the complete set of -edge connected temporal graph patterns . Assume is a connected pattern of edges that is grown from a pattern of edges. Since the three growth options are all possible ways to keep patterns connected during growth, if cannot be covered by growing patterns in , it implies , that is, is not connected, which contradicts with the assumption that is connected (T-connected). Therefore, the completeness also holds for .
Since both the basis and inductive step have been performed, by mathematical induction, the completeness holds.
In sum, we have proved Theorem 1.
Appendix E Proof of Proposition 1
Let and be temporal graph patterns and be a set of temporal graphs. Assume we have with two distinct node mappings and , and . For any such that and matches , we can find two distincted subgraphs in that match because of and . Therefore, , which contradicts with the fact that . Therefore, we have proved the proposition holds.
Appendix F Proof of Lemma 4
Let and be temporal graph patterns, where is discovered before and they satisfy the conditions in subgraph pruning.
First, since the conditions in subgraph pruning are satisfied, we can derive the following facts: (1) and (2) pattern growth in ’s branch will never touch the nodes that cannot map to any nodes in as .
Second, we prove none of the patterns in ’s branch will be the most discriminative, if the conditions in subgraph pruning are satisfied and none of the patterns in ’s branch is the most discriminative. Assume there exists a pattern whose discriminative score is no less than and is the sequence of consecutive growth that grows into . Since no pattern growth in ’s branch will touch the nodes that cannot map to any nodes in , we can safely claim also indicates a valid sequence of consecutive growth (with some timestamp shift) that grows into by Proposition 1. By and , we can infer . It is not hard to see . Thus, we have . Therefore, we can infer , which means is one of the most discriminative patterns which contradicts with the condition that none of the patterns in ’s branch is the most discriminative. Therefore, we can claim any patterns in ’s branch will have discriminative score less than , and the branch can be safely pruned.
In all, we have proved Lemma 4 holds.
Appendix G Proof of Proposition 2
Let and be temporal graph patterns, where is discovered before