Flexible Bayesian Nonlinear Model Configuration

03/05/2020
by   Aliaksandr Hubin, et al.
73

Regression models are used in a wide range of applications providing a powerful scientific tool for researchers from different fields. Linear models are often not sufficient to describe the complex relationship between input variables and a response. This relationship can be better described by non-linearities and complex functional interactions. Deep learning models have been extremely successful in terms of prediction although they are often difficult to specify and potentially suffer from overfitting. In this paper, we introduce a class of Bayesian generalized nonlinear regression models with a comprehensive non-linear feature space. Non-linear features are generated hierarchically, similarly to deep learning, but have additional flexibility on the possible types of features to be considered. This flexibility, combined with variable selection, allows us to find a small set of important features and thereby more interpretable models. A genetically modified Markov chain Monte Carlo algorithm is developed to make inference. Model averaging is also possible within our framework. In various applications, we illustrate how our approach is used to obtain meaningful non-linear models. Additionally, we compare its predictive performance with a number of machine learning algorithms.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/06/2018

Deep Bayesian regression models

Regression models are used for inference and prediction in a wide range ...
07/23/2021

LocalGLMnet: interpretable deep learning for tabular data

Deep learning models have gained great popularity in statistical modelin...
03/07/2017

Regularising Non-linear Models Using Feature Side-information

Very often features come with their own vectorial descriptions which pro...
08/05/2015

Bayesian Approximate Kernel Regression with Variable Selection

Nonlinear kernel regression models are often used in statistics and mach...
04/26/2021

A Survey Of Regression Algorithms And Connections With Deep Learning

Regression has attracted immense interest lately due to its effectivenes...
07/26/2022

A Bayesian hierarchical framework for emulating a complex crop yield simulator

Emulation of complex computer simulations have become an effective tool ...
02/22/2021

You Only Compress Once: Optimal Data Compression for Estimating Linear Models

Linear models are used in online decision making, such as in machine lea...