How Many Machines Can We Use in Parallel Computing for Kernel Ridge Regression?

05/25/2018
by   Meimei Liu, et al.
0

This paper attempts to solve a basic problem in distributed statistical inference: how many machines can we use in parallel computing? In kernel ridge regression, we address this question in two important settings: nonparametric estimation and hypothesis testing. Specifically, we find a range for the number of machines under which optimal estimation/testing is achievable. The employed empirical processes method provides a unified framework, that allows us to handle various regression problems (such as thin-plate splines and nonparametric additive regression) under different settings (such as univariate, multivariate and diverging-dimensional designs). It is worth noting that the upper bounds of the number of machines are proven to be un-improvable (up to a logarithmic factor) in two important cases: smoothing spline regression and Gaussian RKHS regression. Our theoretical findings are backed by thorough numerical studies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2021

Online nonparametric regression with Sobolev kernels

In this work we investigate the variation of the online kernelized ridge...
research
08/26/2023

Estimation and Hypothesis Testing of Derivatives in Smoothing Spline ANOVA Models

This article studies the derivatives in models that flexibly characteriz...
research
06/03/2021

Minimax Optimal Regression over Sobolev Spaces via Laplacian Regularization on Neighborhood Graphs

In this paper we study the statistical properties of Laplacian smoothing...
research
02/05/2019

Optimal Nonparametric Inference via Deep Neural Network

Deep neural network is a state-of-art method in modern science and techn...
research
01/05/2017

Overlapping Cover Local Regression Machines

We present the Overlapping Domain Cover (ODC) notion for kernel machines...
research
05/05/2018

Decentralized Nonparametric Multiple Testing

Consider a big data multiple testing task, where, due to storage and com...
research
05/25/2018

Function Estimation via Reconstruction

This paper introduces an interpolation-based method, called the reconstr...

Please sign up or login with your details

Forgot password? Click here to reset