A non-asymptotic penalization criterion for model selection in mixture of experts models

04/06/2021
by   TrungTin Nguyen, et al.
1

Mixture of experts (MoE) is a popular class of models in statistics and machine learning that has sustained attention over the years, due to its flexibility and effectiveness. We consider the Gaussian-gated localized MoE (GLoME) regression model for modeling heterogeneous data. This model poses challenging questions with respect to the statistical estimation and model selection problems, including feature selection, both from the computational and theoretical points of view. We study the problem of estimating the number of components of the GLoME model, in a penalized maximum likelihood estimation framework. We provide a lower bound on the penalty that ensures a weak oracle inequality is satisfied by our estimator. To support our theoretical result, we perform numerical experiments on simulated and real data, which illustrate the performance of our finite-sample oracle inequality.

READ FULL TEXT

page 12

page 19

research
04/18/2021

Non-asymptotic model selection in block-diagonal mixture of polynomial experts models

Model selection, via penalized likelihood type criteria, is a standard t...
research
09/22/2020

An l_1-oracle inequality for the Lasso in mixture-of-experts regression models

Mixture-of-experts (MoE) models are a popular framework for modeling het...
research
12/03/2019

Bayesian Model Selection for Change Point Detection and Clustering

We address the new problem of estimating a piece-wise constant signal wi...
research
07/14/2019

Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models

Mixtures-of-Experts (MoE) are conditional mixture models that have shown...
research
12/29/2013

A model selection approach for clustering a multinomial sequence with non-negative factorization

We consider a problem of clustering a sequence of multinomial observatio...
research
10/28/2019

Penalized quasi likelihood estimation for variable selection

Penalized methods are applied to quasi likelihood analysis for stochasti...
research
10/12/2020

Robust Finite Mixture Regression for Heterogeneous Targets

Finite Mixture Regression (FMR) refers to the mixture modeling scheme wh...

Please sign up or login with your details

Forgot password? Click here to reset