RisGraph: A Real-Time Streaming System for Evolving Graphs
share
Graphs in the real world are constantly changing and of large scale. In processing these evolving graphs, the combination of update workloads (updating vertices and edges in a streaming manner) and analytical (performing graph algorithms incrementally) workloads is ubiquitous. Throughput, latency, and granularity are three key requirements in processing evolving graphs with such combined workloads. Although there are several streaming systems proposed for evolving graphs to improve latency. They usually use batch-update model to improve throughput but hurt granularity. It is still challenging to fulfill all the requirements simultaneously, especially for power-law graphs because they are difficult to be partitioned. We analyze the computational cost on synthesized power-law graphs and realistic evolving graphs from public datasets. We find that the affected areas are usually small for each update, and there are scheduling opportunities for combined workloads. Based on these observations, we design a real-time streaming system for incremental graph computing called RisGraph. Our novel design on scheduling, trade-offs on data structures and the computing engine make RisGraph satisfy the three requirements at the same time. The evaluation shows RisGraph can ingest millions of updates per second and its 99.9 latency is within 20 milliseconds for graphs with hundreds of millions of vertices and billions of edges on a single commodity machine.
READ FULL TEXT