Mining Massive Hierarchical Data Using a Scalable Probabilistic Graphical Model

12/28/2015
by   Khalifeh Aljadda, et al.
0

Probabilistic Graphical Models (PGM) are very useful in the fields of machine learning and data mining. The crucial limitation of those models,however, is the scalability. The Bayesian Network, which is one of the most common PGMs used in machine learning and data mining, demonstrates this limitation when the training data consists of random variables, each of them has a large set of possible values. In the big data era, one would expect new extensions to the existing PGMs to handle the massive amount of data produced these days by computers, sensors and other electronic devices. With hierarchical data - data that is arranged in a treelike structure with several levels - one would expect to see hundreds of thousands or millions of values distributed over even just a small number of levels. When modeling this kind of hierarchical data across large data sets, Bayesian Networks become infeasible for representing the probability distributions. In this paper we introduce an extension to Bayesian Networks to handle massive sets of hierarchical data in a reasonable amount of time and space. The proposed model achieves perfect precision of 1.0 and high recall of 0.93 when it is used as multi-label classifier for the annotation of mass spectrometry data. On another data set of 1.5 billion search logs provided by CareerBuilder.com the model was able to predict latent semantic relationships between search keywords with accuracy up to 0.80.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/21/2014

PGMHD: A Scalable Probabilistic Graphical Model for Massive Hierarchical Data Problems

In the big data era, scalability has become a crucial requirement for an...
research
04/04/2017

AMIDST: a Java Toolbox for Scalable Probabilistic Machine Learning

The AMIDST Toolbox is a software for scalable probabilistic machine lear...
research
09/29/2017

Toward Scalable Machine Learning and Data Mining: the Bioinformatics Case

In an effort to overcome the data deluge in computational biology and bi...
research
08/10/2011

A survey on independence-based Markov networks learning

This work reports the most relevant technical aspects in the problem of ...
research
03/11/2017

Learning Large-Scale Bayesian Networks with the sparsebn Package

Learning graphical models from data is an important problem with wide ap...
research
05/18/2017

Scalable Exact Parent Sets Identification in Bayesian Networks Learning with Apache Spark

In Machine Learning, the parent set identification problem is to find a ...
research
07/08/2016

Translating Bayesian Networks into Entity Relationship Models, Extended Version

Big data analytics applications drive the convergence of data management...

Please sign up or login with your details

Forgot password? Click here to reset