Regression modelling with I-priors

07/30/2020
by   Wicher Bergsma, et al.
0

We introduce the I-prior methodology as a unifying framework for estimating a variety of regression models, including varying coefficient, multilevel, longitudinal models, and models with functional covariates and responses. It can also be used for multi-class classification, with low or high dimensional covariates. The I-prior is generally defined as a maximum entropy prior. For a regression function, the I-prior is Gaussian with covariance kernel proportional to the Fisher information on the regression function, which is estimated by its posterior distribution under the I-prior. The I-prior has the intuitively appealing property that the more information is available on a linear functional of the regression function, the larger the prior variance, and the smaller the influence of the prior mean on the posterior distribution. Advantages compared to competing methods, such as Gaussian process regression or Tikhonov regularization, are ease of estimation and model comparison. In particular, we develop an EM algorithm with a simple E and M step for estimating hyperparameters, facilitating estimation for complex models. We also propose a novel parsimonious model formulation, requiring a single scale parameter for each (possibly multidimensional) covariate and no further parameters for interaction effects. This simplifies estimation because fewer hyperparameters need to be estimated, and also simplifies model comparison of models with the same covariates but different interaction effects; in this case, the model with the highest estimated likelihood can be selected. Using a number of widely analyzed real data sets we show that predictive performance of our methodology is competitive. An R-package implementing the methodology is available (Jamil, 2019).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/30/2019

iprior: An R Package for Regression Modelling using I-priors

This is an overview of the R package iprior, which implements a unified ...
research
01/30/2021

Gaussian Process for Functional Data Analysis: The GPFDA Package for R

We present and describe the GPFDA package for R. The package provides fl...
research
05/25/2016

How priors of initial hyperparameters affect Gaussian process regression models

The hyperparameters in Gaussian process regression (GPR) model with a sp...
research
01/06/2021

Joint Variable Selection of both Fixed and Random Effects for Gaussian Process-based Spatially Varying Coefficient Models

Spatially varying coefficient (SVC) models are a type of regression mode...
research
06/01/2020

Distributed Bayesian Varying Coefficient Modeling Using a Gaussian Process Prior

Varying coefficient models (VCMs) are widely used for estimating nonline...
research
05/24/2020

Longitudinal Deep Kernel Gaussian Process Regression

We consider the problem of learning predictive models from longitudinal ...

Please sign up or login with your details

Forgot password? Click here to reset