Communication-free Massively Distributed Graph Generation

10/20/2017
by   Daniel Funke, et al.
0

Analyzing massive complex networks yields promising insights about our everyday lives. Building scalable algorithms to do that is a challenging task that requires a careful analysis and extensive evaluation. However, engineering such algorithms is often hindered by the scarcity of publicly available datasets. Network generators serve as a tool to alleviate this problem by providing synthetic instances with controllable parameters. However, many network generators fail to provide instances on a massive scale due to their sequential nature or resource constraints. Additionally, truly scalable network generators are few and often limited in their realism. In this work, we present novel generators for a variety of network models commonly found in practice. By making use of pseudorandomization and divide-and-conquer schemes, our generators follow a communication-free paradigm, i.e. they require no communication. The resulting generators are often embarrassingly parallel and have a near optimal scaling behavior. Overall, we are able to generate instances of up to 2^43 vertices and 2^47 edges in less than 22 minutes on 32768 processors. Therefore, our generators allow new graph families to be used on an unprecedented scale.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/04/2018

Design, Generation, and Validation of Extreme Scale Power-Law Graphs

Massive power-law graphs drive many fields: metagenomics, brain mapping,...
research
03/02/2020

Recent Advances in Scalable Network Generation

Random graph models are frequently used as a controllable and versatile ...
research
05/01/2021

WfChef: Automated Generation of Accurate Scientific Workflow Generators

Scientific workflow applications have become mainstream and their automa...
research
06/07/2020

Deep Graph Generators

Learning distributions of graphs can be used for automatic drug discover...
research
05/16/2019

Efficiently Generating Geometric Inhomogeneous and Hyperbolic Random Graphs

Hyperbolic random graphs (HRG) and geometric inhomogeneous random graphs...
research
10/26/2022

Visually Improved Erosion Algorithm for the Procedural Generation of Tile-based Terrain

Procedural terrain generation is the process of generating a digital rep...

Please sign up or login with your details

Forgot password? Click here to reset