Marius: Learning Massive Graph Embeddings on a Single Machine

01/20/2021
by   Jason Mohoney, et al.
0

We propose a new framework for computing the embeddings of large-scale graphs on a single machine. A graph embedding is a fixed length vector representation for each node (and/or edge-type) in a graph and has emerged as the de-facto approach to apply modern machine learning on graphs. We identify that current systems for learning the embeddings of large-scale graphs are bottlenecked by data movement, which results in poor resource utilization and inefficient training. These limitations require state-of-the-art systems to distribute training across multiple machines. We propose Marius, a system for efficient training of graph embeddings that leverages partition caching and buffer-aware data orderings to minimize disk access and interleaves data movement with computation to maximize utilization. We compare Marius against two state-of-the-art industrial systems on a diverse array of benchmarks. We demonstrate that Marius achieves the same level of accuracy but is up to one order of magnitude faster. We also show that Marius can scale training to datasets an order of magnitude beyond a single machine's GPU and CPU memory capacity, enabling training of configurations with more than a billion edges and 550 GB of total parameters on a single machine with 16 GB of GPU memory and 64 GB of CPU memory. Marius is open-sourced at www.marius-project.org.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/13/2021

Scalable Graph Embedding LearningOn A Single GPU

Graph embedding techniques have attracted growing interest since they co...
research
11/15/2021

AutoGMap: Learning to Map Large-scale Sparse Graphs on Memristive Crossbars

The sparse representation of graphs has shown its great potential for ac...
research
10/19/2021

Boosting Graph Embedding on a Single GPU

Graphs are ubiquitous, and they can model unique characteristics and com...
research
08/17/2017

Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems

We propose a generic algorithmic building block to accelerate training o...
research
02/04/2022

Marius++: Large-Scale Training of Graph Neural Networks on a Single Machine

Graph Neural Networks (GNNs) have emerged as a powerful model for ML ove...
research
08/27/2020

GOSH: Embedding Big Graphs on Small Hardware

In graph embedding, the connectivity information of a graph is used to r...
research
03/07/2019

Computing Three-dimensional Constrained Delaunay Refinement Using the GPU

We propose the first GPU algorithm for the 3D triangulation refinement p...

Please sign up or login with your details

Forgot password? Click here to reset