Cache-Efficient Fork-Processing Patterns on Large Graphs

03/27/2021
by   Shengliang Lu, et al.
0

As large graph processing emerges, we observe a costly fork-processing pattern (FPP) that is common in many graph algorithms. The unique feature of the FPP is that it launches many independent queries from different source vertices on the same graph. For example, an algorithm in analyzing the network community profile can execute Personalized PageRanks that start from tens of thousands of source vertices at the same time. We study the efficiency of handling FPPs in state-of-the-art graph processing systems on multi-core architectures. We find that those systems suffer from severe cache miss penalty because of the irregular and uncoordinated memory accesses in processing FPPs. In this paper, we propose ForkGraph, a cache-efficient FPP processing system on multi-core architectures. To improve the cache reuse, we divide the graph into partitions each sized of LLC capacity, and the queries in an FPP are buffered and executed on the partition basis. We further develop efficient intra- and inter-partition execution strategies for efficiency. For intra-partition processing, since the graph partition fits into LLC, we propose to execute each graph query with efficient sequential algorithms (in contrast with parallel algorithms in existing parallel graph processing systems) and present an atomic-free query processing by consolidating contending operations to cache-resident graph partition. For inter-partition processing, we propose yielding and priority-based scheduling, to reduce redundant work in processing. Besides, we theoretically prove that ForkGraph performs the same amount of work, to within a constant factor, as the fastest known sequential algorithms in FPP queries processing, which is work efficient. Our evaluations on real-world graphs show that ForkGraph significantly outperforms state-of-the-art graph processing systems with two orders of magnitude speedups.

READ FULL TEXT
research
06/21/2018

GPOP: A cache- and work-efficient framework for Graph Processing Over Partitions

The past decade has seen development of many shared-memory graph process...
research
10/20/2021

Scheduling of Graph Queries: Controlling Intra- and Inter-query Parallelism for a High System Throughput

The vast amounts of data used in social, business or traffic networks, b...
research
11/17/2017

Loom: Query-aware Partitioning of Online Graphs

As with general graph processing systems, partitioning data over a clust...
research
01/25/2016

Quegel: A General-Purpose Query-Centric Framework for Querying Big Graphs

Pioneered by Google's Pregel, many distributed systems have been develop...
research
05/31/2018

Start Late or Finish Early: A Distributed Graph Processing System with Redundancy Reduction

Graph processing systems are important in the big data domain. However, ...
research
06/04/2018

A Structure-aware Approach for Efficient Graph Processing

With the advent of the big data, graph are processed in an iterative man...
research
11/24/2021

Locality-based Graph Reordering for Processing Speed-Ups and Impact of Diameter

Graph analysis involves a high number of random memory access patterns. ...

Please sign up or login with your details

Forgot password? Click here to reset