Tensor Regression Using Low-rank and Sparse Tucker Decompositions

11/09/2019
by   Talal Ahmed, et al.
16

This paper studies a tensor-structured linear regression model with a scalar response variable and tensor-structured predictors, such that the regression parameters form a tensor of order d (i.e., a d-fold multiway array) in R^n_1 × n_2 ×...× n_d. This work focuses on the task of estimating the regression tensor from m realizations of the response variable and the predictors where m≪ n = ∏_i n_i. Despite the ill-posedness of this estimation problem, it can still be solved if the parameter tensor belongs to the space of sparse, low Tucker-rank tensors. Accordingly, the estimation procedure is posed as a non-convex optimization program over the space of sparse, low Tucker-rank tensors, and a tensor variant of projected gradient descent is proposed to solve the resulting non-convex problem. In addition, mathematical guarantees are provided that establish the proposed method converges to the correct solution under the right set of conditions. Further, an upper bound on sample complexity of tensor parameter estimation for the model under consideration is characterized for the special case when the individual (scalar) predictors independently draw values from a sub-Gaussian distribution. The sample complexity bound is shown to have a polylogarithmic dependence on n̅ = max{n_i: i∈{1,2,...,d }} and, orderwise, it matches the bound one can obtain from a heuristic parameter counting argument. Finally, numerical experiments demonstrate the efficacy of the proposed tensor model and estimation method on a synthetic dataset and a neuroimaging dataset pertaining to attention deficit hyperactivity disorder. Specifically, the proposed method exhibits better sample complexities on both synthetic and real datasets, demonstrating the usefulness of the model and the method in settings where n ≫ m.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/31/2019

Sparse Tensor Additive Regression

Tensors are becoming prevalent in modern applications such as medical im...
09/15/2016

STORE: Sparse Tensor Response Regression and Neuroimaging Analysis

Motivated by applications in neuroimaging analysis, we propose a new reg...
01/29/2018

Sparse and Low-rank Tensor Estimation via Cubic Sketchings

In this paper, we propose a general framework for sparse and low-rank te...
10/07/2018

Network Response Regression for Modeling Population of Networks with Covariates

Multiple-network data are fast emerging in recent years, where a separat...
07/07/2014

Spectral norm of random tensors

We show that the spectral norm of a random n_1× n_2×...× n_K tensor (or ...
02/21/2017

Scalar and Tensor Parameters for Importing Tensor Index Notation including Einstein Summation Notation

In this paper, we propose a method for importing tensor index notation, ...
08/15/2020

An Augmented Regression Model for Tensors with Missing Values

Heterogeneous but complementary sources of data provide an unprecedented...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.