Flexible Bayesian Nonlinear Model Configuration

by   Aliaksandr Hubin, et al.

Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear models are often not sufficient to describe the complex relationship between input variables and a response. This relationship can be better described by non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. In this paper, we introduce a class of Bayesian generalized nonlinear regression models with a comprehensive non-linear feature space. Non-linear features are generated hierarchically, similarly to deep learning, but have additional flexibility on the possible types of features to be considered. This flexibility, combined with variable selection, allows us to find a small set of important features and thereby more interpretable models. A genetically modified Markov chain Monte Carlo algorithm is developed to make inference. Model averaging is also possible within our framework. In various applications, we illustrate how our approach is used to obtain meaningful non-linear models. Additionally, we compare its predictive performance with a number of machine learning algorithms.


page 1

page 2

page 3

page 4


Deep Bayesian regression models

Regression models are used for inference and prediction in a wide range ...

LocalGLMnet: interpretable deep learning for tabular data

Deep learning models have gained great popularity in statistical modelin...

Regularising Non-linear Models Using Feature Side-information

Very often features come with their own vectorial descriptions which pro...

Bayesian Approximate Kernel Regression with Variable Selection

Nonlinear kernel regression models are often used in statistics and mach...

A Survey Of Regression Algorithms And Connections With Deep Learning

Regression has attracted immense interest lately due to its effectivenes...

A Bayesian hierarchical framework for emulating a complex crop yield simulator

Emulation of complex computer simulations have become an effective tool ...

You Only Compress Once: Optimal Data Compression for Estimating Linear Models

Linear models are used in online decision making, such as in machine lea...