AWAPart: Adaptive Workload-Aware Partitioning of Knowledge Graphs

03/28/2022
by   Amitabh Priyadarshi, et al.
0

Large-scale knowledge graphs are increasingly common in many domains. Their large sizes often exceed the limits of systems storing the graphs in a centralized data store, especially if placed in main memory. To overcome this, large knowledge graphs need to be partitioned into multiple sub-graphs and placed in nodes in a distributed system. But querying these fragmented sub-graphs poses new challenges, such as increased communication costs, due to distributed joins involving cut edges. To combat these problems, a good partitioning should reduce the edge cuts while considering a given query workload. However, a partitioned graph needs to be continually re-partitioned to accommodate changes in the query workload and maintain a good average processing time. In this paper, an adaptive partitioning method for large-scale knowledge graphs is introduced, which adapts the partitioning in response to changes in the query workload. Our evaluation demonstrates that the performance of processing time for queries is improved after dynamically adapting the partitioning of knowledge graph triples.

READ FULL TEXT
research
03/28/2022

WawPart: Workload-Aware Partitioning of Knowledge Graphs

Large-scale datasets in the form of knowledge graphs are often used in n...
research
02/16/2018

PRoST: Distributed Execution of SPARQL Queries Using Mixed Partitioning Strategies

The rapidly growing size of RDF graphs in recent years necessitates dist...
research
11/26/2019

Prediction of Horizontal Data Partitioning Through Query Execution Cost Estimation

The excessively increased volume of data in modern data management syste...
research
01/24/2020

Adaptive Low-level Storage of Very Large Knowledge Graphs

The increasing availability and usage of Knowledge Graphs (KGs) on the W...
research
07/02/2021

On-Demand and Lightweight Knowledge Graph Generation – a Demonstration with DBpedia

Modern large-scale knowledge graphs, such as DBpedia, are datasets which...
research
08/31/2022

The Lothbrok approach for SPARQL Query Optimization over Decentralized Knowledge Graphs

While the Web of Data in principle offers access to a wide range of inte...
research
09/26/2017

PMV: Pre-partitioned Generalized Matrix-Vector Multiplication for Scalable Graph Mining

How can we analyze enormous networks including the Web and social networ...

Please sign up or login with your details

Forgot password? Click here to reset