Nonlinear generalization of the single index model

02/24/2019
by   Zeljko Kereta, et al.
0

Single index model is a powerful yet simple model, widely used in statistics, machine learning, and other scientific fields. It models the regression function as g(<a,x>), where a is an unknown index vector and x are the features. This paper deals with a nonlinear generalization of this framework to allow for a regressor that uses multiple index vectors, adapting to local changes in the responses. To do so we exploit the conditional distribution over function-driven partitions, and use linear regression to locally estimate index vectors. We then regress by applying a kNN type estimator that uses a localized proxy of the geodesic metric. We present theoretical guarantees for estimation of local index vectors and out-of-sample prediction, and demonstrate the performance of our method with experiments on synthetic and real-world data sets, comparing it with state-of-the-art methods.

READ FULL TEXT
research
03/10/2020

Estimating multi-index models with response-conditional least squares

The multi-index model is a simple yet powerful high-dimensional regressi...
research
06/04/2020

Estimation of Monotone Multi-Index Models

In a multi-index model with k index vectors, the input variables are tra...
research
08/13/2021

Fréchet single index models for object response regression

With the availability of more non-euclidean data objects, statisticians ...
research
02/23/2020

Conditional regression for single-index models

The single-index model is a statistical model for intrinsic regression w...
research
12/19/2012

Feature vector regularization in machine learning

Problems in machine learning (ML) can involve noisy input data, and ML c...
research
04/11/2011

Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression

Generalized Linear Models (GLMs) and Single Index Models (SIMs) provide ...
research
10/19/2020

Estimating Stochastic Linear Combination of Non-linear Regressions Efficiently and Scalably

Recently, many machine learning and statistical models such as non-linea...

Please sign up or login with your details

Forgot password? Click here to reset