Log(Graph): A Near-Optimal High-Performance Graph Representation

by   Maciej Besta, et al.

Today's graphs used in domains such as machine learning or social network analysis may contain hundreds of billions of edges. Yet, they are not necessarily stored efficiently, and standard graph representations such as adjacency lists waste a significant number of bits while graph compression schemes such as WebGraph often require time-consuming decompression. To address this, we propose Log(Graph): a graph representation that combines high compression ratios with very low-overhead decompression to enable cheaper and faster graph processing. The key idea is to encode a graph so that the parts of the representation approach or match the respective storage lower bounds. We call our approach "graph logarithmization" because these bounds are usually logarithmic. Our high-performance Log(Graph) implementation based on modern bitwise operations and state-of-the-art succinct data structures achieves high compression ratios as well as performance. For example, compared to the tuned Graph Algorithm Processing Benchmark Suite (GAPBS), it reduces graph sizes by 20-35 reducing amounts of transferred data. It approaches the compression ratio of the established WebGraph compression library while enabling speedups of up to more than 2x. Log(Graph) can improve the design of various graph processing engines or libraries on single NUMA nodes as well as distributed-memory systems.



page 2

page 7

page 11

page 14

page 17

page 19

page 21


Slim Graph: Practical Lossy Graph Compression for Approximate Graph Processing, Storage, and Analytics

We propose Slim Graph: the first programming model and framework for pra...

Zuckerli: A New Compressed Representation for Graphs

Zuckerli is a scalable compression system meant for large real-world gra...

Graph Compression Using Pattern Matching Techniques

Graphs can be used to represent a wide variety of data belonging to diff...

Optimizing Communication by Compression for Multi-GPU Scalable Breadth-First Searches

The Breadth First Search (BFS) algorithm is the foundation and building ...

Partition and Code: learning how to compress graphs

Can we use machine learning to compress graph data? The absence of order...

Survey and Taxonomy of Lossless Graph Compression and Space-Efficient Graph Representations

Various graphs such as web or social networks may contain up to trillion...

SlimSell: A Vectorizable Graph Representation for Breadth-First Search

Vectorization and GPUs will profoundly change graph processing. Traditio...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.