On Cross-validation for Sparse Reduced Rank Regression

12/30/2018

∙

In high-dimensional data analysis, regularization methods pursuing sparsity and/or low rank have received a lot of attention recently. To provide a proper amount of shrinkage, it is typical to use a grid search and a model comparison criterion to find the optimal regularization parameters. However, we show that fixing the parameters across all folds may result in an inconsistency issue, and it is more appropriate to cross-validate projection-selection patterns to obtain the best coefficient estimate. Our in-sample error studies in jointly sparse and rank-deficient models lead to a new class of information criteria with four scale-free forms to bypass the estimation of the noise level. By use of an identity, we propose a novel scale-free calibration to help cross-validation achieve the minimax optimal error rate non-asymptotically. Experiments support the efficacy of the proposed methods.

READ FULL TEXT

On Cross-validation for Sparse Reduced Rank Regression

A least-squares method for sparse low rank approximation of multivariate functions

Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradients

On High-Dimensional Gaussian Comparisons For Cross-Validation

Cross validating extensions of kernel, sparse or regular partial least squares regression models to censored data

Sparse Reduced-Rank Regression for Simultaneous Rank and Variable Selection via Manifold Optimization

An Empirical Comparison of V-fold Penalisation and Cross Validation for Model Selection in Distribution-Free Regression

No Cross-Validation Required: An Analytical Framework for Regularized Mixed-Integer Problems (Extended Version)

On Cross-validation for Sparse Reduced Rank Regression

Related Research

A least-squares method for sparse low rank approximation of multivariate functions

Towards new cross-validation-based estimators for Gaussian process regression: efficient adjoint computation of gradients

On High-Dimensional Gaussian Comparisons For Cross-Validation

Cross validating extensions of kernel, sparse or regular partial least squares regression models to censored data

Sparse Reduced-Rank Regression for Simultaneous Rank and Variable Selection via Manifold Optimization

An Empirical Comparison of V-fold Penalisation and Cross Validation for Model Selection in Distribution-Free Regression

No Cross-Validation Required: An Analytical Framework for Regularized Mixed-Integer Problems (Extended Version)