DeepAI AI Chat
Log In Sign Up

Query Processing on Large Graphs: Approaches To Scalability and Response Time Trade Offs

05/14/2019
by   Soumyava Das, et al.
The University of Texas at Arlington
Teradata
0

With the advent of social networks and the web, the graph sizes have grown too large to fit in main memory precipitating the need for alternative approaches for an efficient, scalable evaluation of queries on graphs of any size. Here, we use the divide and conquer approach by partitioning a graph and process queries over partitions to obtain all or specified number of answers. This entails correctly computing answers that span multiple partitions or even need the same partition more than once. Given a set of partitions, there are many approaches to evaluate a query: i) One Partition At a Time approach, ii) Traditional use of Multiple Processors, and iii) using the Map/Reduce Multi-Processor approach. Approach (i), detailed in this paper, has established scalability through independent processing of partitions. The other two approaches address response time in addition to scalability. For approach (i), necessary minimal book keeping has been identified and its correctness established in this paper. Query answering on partitioned graphs also requires analyzing partitioning schemes for their impact on query processing and determining the number and the sequence in which partitions need to be loaded to reduce the response time to process queries. We correlate query properties and partition characteristics to reduce query processing time in terms of the resources available. We also identify a set of quantitative metrics and use them to formulate heuristics to determine the order of loading partitions for efficient query processing. For approach (i), experiments on large graphs (synthetic, real-world) using different partitioning schemes analyze the proposed heuristics on a variety of query types. The other two approaches are fleshed out and analyzed. An existing graph querying system has been extended to evaluate queries on partitioned graphs. Finally all approaches are contrasted.

READ FULL TEXT
03/28/2022

WawPart: Workload-Aware Partitioning of Knowledge Graphs

Large-scale datasets in the form of knowledge graphs are often used in n...
12/27/2022

Range-Based Set Reconciliation

Range-based set reconciliation is a simple approach to efficiently compu...
05/30/2018

Q-Graph: Preserving Query Locality in Multi-Query Graph Processing

Arising user-centric graph applications such as route planning and perso...
03/29/2020

Learning and Testing Variable Partitions

Let F be a multivariate function from a product set Σ^n to an Abelian gr...
05/29/2009

Divide and Conquer: Partitioning Online Social Networks

Online Social Networks (OSNs) have exploded in terms of scale and scope ...
02/16/2018

PRoST: Distributed Execution of SPARQL Queries Using Mixed Partitioning Strategies

The rapidly growing size of RDF graphs in recent years necessitates dist...
08/25/2018

Efficiently Processing Workflow Provenance Queries on SPARK

In this paper, we investigate how we can leverage Spark platform for eff...