Pointwise adaptation via stagewise aggregation of local estimates for multiclass classification

04/08/2018
by   Nikita Puchkin, et al.
0

We consider a problem of multiclass classification, where the training sample S_n = {(X_i, Y_i)}_i=1^n is generated from the model p(Y = m | X = x) = θ_m(x), 1 ≤ m ≤ M, and θ_1(x), ..., θ_M(x) are unknown Lipschitz functions. Given a test point X, our goal is to estimate θ_1(X), ..., θ_M(X). An approach based on nonparametric smoothing uses a localization technique, i.e. the weight of observation (X_i, Y_i) depends on the distance between X_i and X. However, local estimates strongly depend on localizing scheme. In our solution we fix several schemes W_1, ..., W_K, compute corresponding local estimates θ^(1), ..., θ^(K) for each of them and apply an aggregation procedure. We propose an algorithm, which constructs a convex combination of the estimates θ^(1), ..., θ^(K) such that the aggregated estimate behaves approximately as well as the best one from the collection θ^(1), ..., θ^(K). We also study theoretical properties of the procedure, prove oracle results and establish rates of convergence under mild assumptions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/16/2021

Aggregating estimates by convex optimization

We discuss the approach to estimate aggregation and adaptive estimation ...
research
12/20/2022

Nonparametric plug-in classifier for multiclass classification of S.D.E. paths

We study the multiclass classification problem where the features come f...
research
10/06/2021

Variance function estimation in regression model via aggregation procedures

In the regression problem, we consider the problem of estimating the var...
research
09/29/2005

Simultaneous adaptation to the margin and to complexity in classification

We consider the problem of adaptation to the margin and to complexity in...
research
01/08/2019

Monotone Least Squares and Isotonic Quantiles

We consider bivariate observations (X_1,Y_1),...,(X_n,Y_n) such that, co...
research
10/10/2012

Sequential Convex Programming Methods for A Class of Structured Nonlinear Programming

In this paper we study a broad class of structured nonlinear programming...
research
01/15/2015

The Fast Convergence of Incremental PCA

We consider a situation in which we see samples in R^d drawn i.i.d. from...

Please sign up or login with your details

Forgot password? Click here to reset