Minimax Optimal Bayesian Aggregation

03/06/2014
by   Yun Yang, et al.
0

It is generally believed that ensemble approaches, which combine multiple algorithms or models, can outperform any single algorithm at machine learning tasks, such as prediction. In this paper, we propose Bayesian convex and linear aggregation approaches motivated by regression applications. We show that the proposed approach is minimax optimal when the true data-generating model is a convex or linear combination of models in the list. Moreover, the method can adapt to sparsity structure in which certain models should receive zero weights, and the method is tuning parameter free unlike competitors. More generally, under an M-open view when the truth falls outside the space of all convex/linear combinations, our theory suggests that the posterior measure tends to concentrate on the best approximation of the truth at the minimax rate. We illustrate the method through simulation studies and several applications.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/01/2015

Optimal Binary Classifier Aggregation for General Losses

We address the problem of aggregating an ensemble of predictors with kno...
research
05/06/2022

Efficient Minimax Optimal Estimators For Multivariate Convex Regression

We study the computational aspects of the task of multivariate convex re...
research
12/08/2009

Hyper-sparse optimal aggregation

In this paper, we consider the problem of "hyper-sparse aggregation". Na...
research
11/19/2019

Minimax rates of ℓ_p-losses for high-dimensional linear regression models with additive measurement errors over ℓ_q-balls

We study minimax rates for high-dimensional linear regression with addit...
research
06/29/2018

Are Condorcet and minimax voting systems the best?

In a search for the best voting system for single winner elections with ...
research
11/22/2022

Bayesian estimation in a multidimensional diffusion model with high frequency data

We consider nonparametric Bayesian inference in a multidimensional diffu...
research
06/08/2021

Minimax and adaptive tests for detecting abrupt and possibly transitory changes in a Poisson process

Motivated by applications in cybersecurity and epidemiology, we consider...

Please sign up or login with your details

Forgot password? Click here to reset