
Dynamic Tensor Clustering
Dynamic tensor data are becoming prevalent in numerous applications. Existing tensor clustering methods either fail to account for the dynamic nature of the data, or are inapplicable to a generalorder tensor. Also there is often a gap between statistical guarantee and computational efficiency for existing tensor clustering solutions. In this article, we aim to bridge this gap by proposing a new dynamic tensor clustering method, which takes into account both sparsity and fusion structures, and enjoys strong statistical guarantees as well as high computational efficiency. Our proposal is based upon a new structured tensor factorization that encourages both sparsity and smoothness in parameters along the specified tensor modes. Computationally, we develop a highly efficient optimization algorithm that benefits from substantial dimension reduction. In theory, we first establish a nonasymptotic error bound for the estimator from the structured tensor factorization. Built upon this error bound, we then derive the rate of convergence of the estimated cluster centers, and show that the estimated clusters recover the true cluster structures with a high probability. Moreover, our proposed method can be naturally extended to coclustering of multiple modes of the tensor data. The efficacy of our approach is illustrated via simulations and a brain dynamic functional connectivity analysis from an Autism spectrum disorder study.
08/24/2017 ∙ by Will Wei Sun, et al. ∙ 0 ∙ shareread it

Stability Enhanced LargeMargin Classifier Selection
Stability is an important aspect of a classification procedure because unstable predictions can potentially reduce users' trust in a classification system and also harm the reproducibility of scientific conclusions. The major goal of our work is to introduce a novel concept of classification instability, i.e., decision boundary instability (DBI), and incorporate it with the generalization error (GE) as a standard for selecting the most accurate and stable classifier. Specifically, we implement a twostage algorithm: (i) initially select a subset of classifiers whose estimated GEs are not significantly different from the minimal estimated GE among all the candidate classifiers; (ii) the optimal classifier is chosen as the one achieving the minimal DBI among the subset selected in stage (i). This general selection principle applies to both linear and nonlinear classifiers. Largemargin classifiers are used as a prototypical example to illustrate the above idea. Our selection method is shown to be consistent in the sense that the optimal classifier simultaneously achieves the minimal GE and the minimal DBI. Various simulations and real examples further demonstrate the advantage of our method over several alternative approaches.
01/20/2017 ∙ by Will Wei Sun, et al. ∙ 0 ∙ shareread it

STORE: Sparse Tensor Response Regression and Neuroimaging Analysis
Motivated by applications in neuroimaging analysis, we propose a new regression model, Sparse TensOr REsponse regression (STORE), with a tensor response and a vector predictor. STORE embeds two key sparse structures: elementwise sparsity and lowrankness. It can handle both a nonsymmetric and a symmetric tensor response, and thus is applicable to both structural and functional neuroimaging data. We formulate the parameter estimation as a nonconvex optimization problem, and develop an efficient alternating updating algorithm. We establish a nonasymptotic estimation error bound for the actual estimator obtained from the proposed algorithm. This error bound reveals an interesting interaction between the computational efficiency and the statistical rate of convergence. When the distribution of the error tensor is Gaussian, we further obtain a fast estimation error rate which allows the tensor dimension to grow exponentially with the sample size. We illustrate the efficacy of our model through intensive simulations and an analysis of the Autism spectrum disorder neuroimaging data.
09/15/2016 ∙ by Will Wei Sun, et al. ∙ 0 ∙ shareread it

Sparse Tensor Graphical Model: Nonconvex Optimization and Statistical Inference
We consider the estimation and inference of sparse graphical models that characterize the dependency structure of highdimensional tensorvalued data. To facilitate the estimation of the precision matrix corresponding to each way of the tensor, we assume the data follow a tensor normal distribution whose covariance has a Kronecker product structure. A critical challenge in the estimation and inference of this model is the fact that its penalized maximum likelihood estimation involves minimizing a nonconvex objective function. To address it, this paper makes two contributions: (i) In spite of the nonconvexity of this estimation problem, we prove that an alternating minimization algorithm, which iteratively estimates each sparse precision matrix while fixing the others, attains an estimator with the optimal statistical rate of convergence. Notably, such an estimator achieves estimation consistency with only one tensor sample, which was not observed in the previous work. (ii) We propose a debiased statistical inference procedure for testing hypotheses on the true support of the sparse precision matrices, and employ it for testing a growing number of hypothesis with false discovery rate (FDR) control. The asymptotic normality of our test statistic and the consistency of FDR control procedure are established. Our theoretical results are further backed up by thorough numerical studies. We implement the methods into a publicly available R package Tlasso.
09/15/2016 ∙ by Will Wei Sun, et al. ∙ 0 ∙ shareread it

Sparse Convex Clustering
Convex clustering, a convex relaxation of kmeans clustering and hierarchical clustering, has drawn recent attentions since it nicely addresses the instability issue of traditional nonconvex clustering methods. Although its computational and statistical properties have been recently studied, the performance of convex clustering has not yet been investigated in the highdimensional clustering scenario, where the data contains a large number of features and many of them carry no information about the clustering structure. In this paper, we demonstrate that the performance of convex clustering could be distorted when the uninformative features are included in the clustering. To overcome it, we introduce a new clustering method, referred to as Sparse Convex Clustering, to simultaneously cluster observations and conduct feature selection. The key idea is to formulate convex clustering in a form of regularization, with an adaptive grouplasso penalty term on cluster centers. In order to optimally balance the tradeoff between the cluster fitting and sparsity, a tuning criterion based on clustering stability is developed. In theory, we provide an unbiased estimator for the degrees of freedom of the proposed sparse convex clustering method. Finally, the effectiveness of the sparse convex clustering is examined through a variety of numerical experiments and a real data application.
01/18/2016 ∙ by Binhuan Wang, et al. ∙ 0 ∙ shareread it

Provable Sparse Tensor Decomposition
We propose a novel sparse tensor decomposition method, namely Tensor Truncated Power (TTP) method, that incorporates variable selection into the estimation of decomposition components. The sparsity is achieved via an efficient truncation step embedded in the tensor power iteration. Our method applies to a broad family of high dimensional latent variable models, including high dimensional Gaussian mixture and mixtures of sparse regressions. A thorough theoretical investigation is further conducted. In particular, we show that the final decomposition estimator is guaranteed to achieve a local statistical rate, and further strengthen it to the global statistical rate by introducing a proper initialization procedure. In high dimensional regimes, the obtained statistical rate significantly improves those shown in the existing nonsparse decomposition methods. The empirical advantages of TTP are confirmed in extensive simulated results and two real applications of clickthrough rate prediction and highdimensional gene clustering.
02/05/2015 ∙ by Will Wei Sun, et al. ∙ 0 ∙ shareread it

MixedEffect TimeVarying Network Model and Application in Brain Connectivity Analysis
Timevarying networks are fast emerging in a wide range of scientific and business disciplines. Most existing dynamic network models are limited to a singlesubject and discretetime setting. In this article, we propose a mixedeffect multisubject continuoustime stochastic blockmodel that characterizes the timevarying behavior of the network at the population level, meanwhile taking into account individual subject variability. We develop a multistep optimization procedure for a constrained stochastic blockmodel estimation, and derive the asymptotic property of the estimator. We demonstrate the effectiveness of our method through both simulations and an application to a study of brain development in youth.
06/11/2018 ∙ by Jingfei Zhang, et al. ∙ 0 ∙ shareread it

Provable Convex Coclustering of Tensors
Cluster analysis is a fundamental tool for pattern discovery of complex heterogeneous data. Prevalent clustering methods mainly focus on vector or matrixvariate data and are not applicable to generalorder tensors, which arise frequently in modern scientific and business applications. Moreover, there is a gap between statistical guarantees and computational efficiency for existing tensor clustering solutions due to the nature of their nonconvex formulations. In this work, we bridge this gap by developing a provable convex formulation of tensor coclustering. Our convex coclustering (CoCo) estimator enjoys stability guarantees and is both computationally and storage efficient. We further establish a nonasymptotic error bound for the CoCo estimator, which reveals a surprising "blessing of dimensionality" phenomenon that does not exist in vector or matrixvariate cluster analysis. Our theoretical findings are supported by extensive simulated studies. Finally, we apply the CoCo estimator to the cluster analysis of advertisement click tensor data from a major online company. Our clustering results provide meaningful business insights to improve advertising effectiveness.
03/17/2018 ∙ by Eric C. Chi, et al. ∙ 0 ∙ shareread it

Network Response Regression for Modeling Population of Networks with Covariates
Multiplenetwork data are fast emerging in recent years, where a separate network over a common set of nodes is measured for each individual subject, along with rich subject covariates information. Existing network analysis methods have primarily focused on modeling a single network, and are not directly applicable to multiple networks with subject covariates. In this article, we propose a new network response regression model, where the observed networks are treated as matrixvalued responses, and the individual covariates as predictors. The new model characterizes the populationlevel connectivity pattern through a lowrank intercept matrix, and the parsimonious effects of subject covariates on the network through a sparse slope tensor. We formulate the parameter estimation as a nonconvex optimization problem, and develop an efficient alternating gradient descent algorithm. We establish the nonasymptotic error bound for the actual estimator from our optimization algorithm. Built upon this error bound, we derive the strong consistency for network community recovery, as well as the edge selection consistency. We demonstrate the efficacy of our method through intensive simulations and two brain connectivity studies.
10/07/2018 ∙ by Jingfei Zhang, et al. ∙ 0 ∙ shareread it

Sparse Tensor Additive Regression
Tensors are becoming prevalent in modern applications such as medical imaging and digital marketing. In this paper, we propose a sparse tensor additive regression (STAR) that models a scalar response as a flexible nonparametric function of tensor covariates. The proposed model effectively exploits the sparse and lowrank structures in the tensor additive regression. We formulate the parameter estimation as a nonconvex optimization problem, and propose an efficient penalized alternating minimization algorithm. We establish a nonasymptotic error bound for the estimator obtained from each iteration of the proposed algorithm, which reveals an interplay between the optimization error and the statistical rate of convergence. We demonstrate the efficacy of STAR through extensive comparative simulation studies, and an application to the clickthroughrate prediction in online advertising.
03/31/2019 ∙ by Botao Hao, et al. ∙ 0 ∙ shareread it
Will Wei Sun
is this you? claim profile