
Additive Gaussian Processes
We introduce a Gaussian process model of functions which are additive. A...
read it

Projection Pursuit Gaussian Process Regression
A primary goal of computer experiments is to reconstruct the function gi...
read it

PostRegularization Confidence Bands for High Dimensional Nonparametric Models with Local Sparsity
We propose a novel high dimensional nonparametric model named ATLAS whic...
read it

On lower bounds for the biasvariance tradeoff
It is a common phenomenon that for highdimensional and nonparametric st...
read it

Efficient estimation of the ANOVA mean dimension, with an application to neural net classification
The mean dimension of a black box function of d variables is a convenien...
read it

Convexconstrained Sparse Additive Modeling and Its Extensions
Sparse additive modeling is a class of effective methods for performing ...
read it

Approximating Constraint Satisfaction Problems on HighDimensional Expanders
We consider the problem of approximately solving constraint satisfaction...
read it
Additive Approximations in High Dimensional Nonparametric Regression via the SALSA
High dimensional nonparametric regression is an inherently difficult problem with known lower bounds depending exponentially in dimension. A popular strategy to alleviate this curse of dimensionality has been to use additive models of first order, which model the regression function as a sum of independent functions on each dimension. Though useful in controlling the variance of the estimate, such models are often too restrictive in practical settings. Between nonadditive models which often have large variance and first order additive models which have large bias, there has been little work to exploit the tradeoff in the middle via additive models of intermediate order. In this work, we propose SALSA, which bridges this gap by allowing interactions between variables, but controls model capacity by limiting the order of interactions. SALSA minimises the residual sum of squares with squared RKHS norm penalties. Algorithmically, it can be viewed as Kernel Ridge Regression with an additive kernel. When the regression function is additive, the excess risk is only polynomial in dimension. Using the GirardNewton formulae, we efficiently sum over a combinatorial number of terms in the additive expansion. Via a comparison on 15 real datasets, we show that our method is competitive against 21 other alternatives.
READ FULL TEXT
Comments
There are no comments yet.