Combining Gradient Boosting Machines with Collective Inference to Predict Continuous Values

by   Iman Alodah, et al.
Purdue University

Gradient boosting of regression trees is a competitive procedure for learning predictive models of continuous data that fits the data with an additive non-parametric model. The classic version of gradient boosting assumes that the data is independent and identically distributed. However, relational data with interdependent, linked instances is now common and the dependencies in such data can be exploited to improve predictive performance. Collective inference is one approach to exploit relational correlation patterns and significantly reduce classification error. However, much of the work on collective learning and inference has focused on discrete prediction tasks rather than continuous. this work, we investigate how to combine these two paradigms together to improve regression in relational domains. Specifically, we propose a boosting algorithm for learning a collective inference model that predicts a continuous target variable. In the algorithm, we learn a basic relational model, collectively infer the target values, and then iteratively learn relational models to predict the residuals. We evaluate our proposed algorithm on a real network dataset and show that it outperforms alternative boosting methods. However, our investigation also revealed that the relational features interact together to produce better predictions.


page 1

page 2

page 3

page 4


Relational Boosted Regression Trees

Many tasks use data housed in relational databases to train boosted regr...

A Comparative Study of Social Network Classifiers for Predicting Churn in the Telecommunication Industry

Relational learning in networked data has been shown to be effective in ...

Column Networks for Collective Classification

Relational learning deals with data that are characterized by relational...

Collective Entity Disambiguation with Structured Gradient Tree Boosting

We present a gradient-tree-boosting-based structured learning model for ...

Lifted Inference for Relational Continuous Models

Relational Continuous Models (RCMs) represent joint probability densitie...

Graph Based Relational Features for Collective Classification

Statistical Relational Learning (SRL) methods have shown that classifica...

Please sign up or login with your details

Forgot password? Click here to reset