Log In Sign Up

Hierarchical Mixtures-of-Experts for Exponential Family Regression Models with Generalized Linear Mean Functions: A Survey of Approximation and Consistency Results

by   Wenxin Jiang, et al.

We investigate a class of hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form psi(ga+fx^Tfgb) are mixed. Here psi(...) is the inverse link function. Suppose the true response y follows an exponential family regression model with mean function belonging to a class of smooth functions of the form psi(h(fx)) where h(...)in W_2^infty (a Sobolev class over [0,1]^s). It is shown that the HME probability density functions can approximate the true density, at a rate of O(m^-2/s) in L_p norm, and at a rate of O(m^-4/s) in Kullback-Leibler divergence. These rates can be achieved within the family of HME structures with no more than s-layers, where s is the dimension of the predictor fx. It is also shown that likelihood-based inference based on HME is consistent in recovering the truth, in the sense that as the sample size n and the number of experts m both increase, the mean square error of the predicted mean response goes to zero. Conditions for such results to hold are stated and discussed.


page 1

page 2

page 3

page 7


Convergence Rates for Mixture-of-Experts

In mixtures-of-experts (ME) model, where a number of submodels (experts)...

Online Algorithms For Parameter Mean And Variance Estimation In Dynamic Regression Models

We study the problem of estimating the parameters of a regression model ...

Some Results on Tighter Bayesian Lower Bounds on the Mean-Square Error

In random parameter estimation, Bayesian lower bounds (BLBs) for the mea...

Approximation Methods for Mixed Models with Probit Link Functions

We study approximation methods for a large class of mixed models with a ...

Best Linear Predictor with Missing Response: Locally Robust Approach

This paper provides asymptotic theory for Inverse Probability Weighing (...

Variational Hierarchical Mixtures for Learning Probabilistic Inverse Dynamics

Well-calibrated probabilistic regression models are a crucial learning c...