On Low-rank Trace Regression under General Sampling Distribution

04/18/2019
by   Nima Hamidi, et al.
0

A growing number of modern statistical learning problems involve estimating a large number of parameters from a (smaller) number of observations. In a subset of these problems (matrix completion, matrix compressed sensing, and multi-task learning) the unknown parameters form a high-dimensional matrix, and two popular approaches for the estimation are trace-norm regularized linear regression or alternating minimization. It is also known that these estimators satisfy certain optimal tail bounds under assumptions on rank, coherence, or spikiness of the unknown matrix. We study a general family of estimators and sampling distribution that include the above two estimators, and introduce a general notion of spikiness and rank for the unknown matrix. Next, we extend the existing literature on the analysis of these estimators and provide a unifying technique to prove tail bounds for the estimation error. We demonstrate the benefit of this generalization by studying its application to four problems of (1) matrix completion, (2) multi-task learning, (3) compressed sensing with Gaussian ensembles, and (4) compressed sensing with factored measurements. For (1) and (3), we recover matching tail bounds as those found in the literature, and for (2) and (4) we obtain (to the best of our knowledge) the first tail bounds. Our approach relies on a generic recipe to prove restricted strong convexity for the sampling operator of the trace regression, that only requires finding upper bounds on certain norms of the parameter matrix.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/04/2021

Nonparametric Trace Regression in High Dimensions via Sign Series Representation

Learning of matrix-valued data has recently surged in a range of scienti...
research
11/09/2022

A Unified Analysis of Multi-task Functional Linear Regression Models with Manifold Constraint and Composite Quadratic Penalty

This work studies the multi-task functional linear regression models whe...
research
03/29/2016

Unified View of Matrix Completion under General Structural Constraints

In this paper, we present a unified analysis of matrix completion under ...
research
04/23/2015

Regularization-free estimation in trace regression with symmetric positive semidefinite matrices

Over the past few years, trace regression models have received considera...
research
05/13/2015

Optimal linear estimation under unknown nonlinear transform

Linear regression studies the problem of estimating a model parameter β^...
research
02/12/2013

Coherence and sufficient sampling densities for reconstruction in compressed sensing

We give a new, very general, formulation of the compressed sensing probl...
research
08/09/2014

Matrix Coherence and the Nystrom Method

The Nystrom method is an efficient technique used to speed up large-scal...

Please sign up or login with your details

Forgot password? Click here to reset