Characterization of Deterministic and Probabilistic Sampling Patterns for Finite Completability of Low Tensor-Train Rank Tensor

03/22/2017 ∙ by Morteza Ashraphijuo, et al. ∙ 0

In this paper, we analyze the fundamental conditions for low-rank tensor completion given the separation or tensor-train (TT) rank, i.e., ranks of unfoldings. We exploit the algebraic structure of the TT decomposition to obtain the deterministic necessary and sufficient conditions on the locations of the samples to ensure finite completability. Specifically, we propose an algebraic geometric analysis on the TT manifold that can incorporate the whole rank vector simultaneously in contrast to the existing approach based on the Grassmannian manifold that can only incorporate one rank component. Our proposed technique characterizes the algebraic independence of a set of polynomials defined based on the sampling pattern and the TT decomposition, which is instrumental to obtaining the deterministic condition on the sampling pattern for finite completability. In addition, based on the proposed analysis, assuming that the entries of the tensor are sampled independently with probability p, we derive a lower bound on the sampling probability p, or equivalently, the number of sampled entries that ensures finite completability with high probability. Moreover, we also provide the deterministic and probabilistic conditions for unique completability.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Most of the literature on low-rank data completion (either matrix or tensor) propose optimization-based algorithms to construct a completion that matches the given samples and rank. For example, for the two-way tensor, i.e., matrix, many algorithms have been proposed that are based on convex relaxation of rank [1, 2, 3, 4, 5] or alternating minimization [6, 7]. Similarly, for higher dimensional data a number of tensor completion algorithms exist that are based on different convex relaxations of the tensor ranks [8, 9, 10, 11]

or other heuristics

[12, 13, 14, 15, 16, 17]. The low-rank tensor completion problem has various applications, including compressed sensing [18, 19, 8], visual data reconstruction [20, 17], seismic data processing [21, 22, 16], RF fingerprinting [23, 15], reconstruction of cellular data [24], etc..

Existing works on optimization-based matrix or tensor completion usually make a set of strong assumptions on the correlations of the values of either the sampled or non-sampled entries (such as coherence) in order to provide a tensor that approximately fits in the sampled tensor. In contrast, here we are interested in investigating fundamental conditions on the sampling pattern that guarantee the existence of finite or unique number of completions. Such conditions are “fundamental” in the sense that they are independent of either the optimization formulation or the optimization algorithm used to compute the completion. The matrix version of this problem has been treated in [25] and [26] for single-view and multi-view data, respectively. Also, the tensor version of this problem under the Tucker rank has been treated in [27]. In this paper, we investigate this problem for tensors under the tensor-train (TT) rank.

There are a number of tensor decompositions available, including Tucker decomposition or higher-order singular value decomposition

[28, 29, 30], polyadic decomposition [31, 32], tubal rank decomposition [33] and several other representations [34, 35, 36]. TT decomposition (also known as tree-train decomposition) was proposed in the field of quantum physics about years ago [37, 38]

. Later it was used in the area of machine learning

[39, 40, 41]. A comprehensive survey on TT decomposition and the manifold of tensors of fixed TT rank can be found in [42] that also includes a comparison between the TT and Tucker decompositions for a better understanding of the advantages of TT decomposition.

In this paper, we propose a geometric analysis on the TT manifold to study the problem of low-rank tensor completion given the TT rank. We first briefly mention the differences and new challenges for the TT model in comparison with the Tucker model considered in [27]. In Tucker decomposition which is the high-order singular value decomposition, we have a -way tensor as the core which is the generalization of the basis for matrices. However, in TT decomposition we are dealing with two and three-way tensors and besides, the geometries of the Tucker and TT manifolds are totally different. Moreover, the notions of tensor multiplications in these two decompositions are different, and therefore the polynomials that can be obtained through each observed entry have completely different structures (to study algebraic independence).

Let denote the sampled tensor and denote the binary sampling pattern tensor that is of the same dimension and size as . The entries of that correspond to the observed entries of are equal to and the rest of the entries are set as . This paper is mainly concerned with the following three problems.

Problem (i): Given the TT rank, characterize the necessary and sufficient conditions on the sampling pattern , under which there exist only finitely many completions of .

We define a polynomial for each sampled entry such that the variables of the polynomial are the entries of the two or three-dimensional tensors in the TT decomposition. Then, we propose a geometric method on the TT manifold to obtain the maximum number algebraically independent polynomials (among all the defined polynomials for any of the sampled entries) in terms of the geometric structure of the sampling pattern . Finally, we show that if the maximum number algebraically independent polynomials meets a threshold, which depends on the structure of the sampling pattern , the sampled tensor is finitely completable. We emphasize the fact that the proposed algebraic geometry analysis on the TT manifold is not a simple generalization of the existing analysis on the Grassmannian or Tucker manifold as almost every step needs to be developed anew.

Problem (ii): Given the TT rank, characterize a sufficient conditions on the sampling pattern , under which there exists only one completion of .

We use the developed tools for solving Problem (i) and in addition to the condition for finite completability, we add more polynomials (samples) in a way such that the corresponding minimally algebraically dependent set of polynomials leads to that all involved variables can be determined uniquely.

Problem (iii): Provide a lower bound on the total number of sampled entries such that the proposed conditions on the sampling pattern for finite and unique completability are satisfied with high probability.

Assuming that the entries of are sampled independently with probability , we develop lower bounds on such that the deterministic conditions for Problems (i) and (ii) are met with high probability.

The remainder of this paper is organized as follows. In Section II, the preliminaries and problem statement are presented. Problems (i), (ii) and (iii) are treated in Sections III, IV and V, respectively. Some numerical results are provided in Section VI. Finally, Section VII concludes the paper.

Ii Background

Ii-a Preliminaries and Notations

In this paper, it is assumed that a -way tensor is sampled. For the sake of simplicity in notation, define and . Also, for any real number , define .

Define the matrix as the -th unfolding of the tensor , such that , where and are two bijective mappings and represents an entry of tensor with coordinate .

The separation or tensor-train (TT) rank of a tensor is defined as where , . Note that in general and also is simply the conventional matrix rank when . The TT decomposition of a tensor is given by

(1)

where the -way tensors for and matrices and are the components of this decomposition and furthermore the tensor product in (1) is defined as

(2)

Observe that the above tensor multiplication for two tensors and is similar to the simple matrix multiplication across the -th dimension of and the first dimension of . Hence, , and also . For notational simplicity, we denote . Given the order and dimension sizes , the space of all tensors of fixed TT rank vector is a manifold of dimension [42]

(3)

where . As and are two-way tensors, we can also denote them by and in this paper.

Denote as the binary sampling pattern tensor that is of the same size as and if is observed and otherwise. denotes the first rows of the matrix and denotes the transpose of .

Let be the -th matricization of the tensor , i.e., the matrix has rows and columns such that , where is a bijective mapping. Observe that for any arbitrary tensor , the first matricization and the first unfolding are the same, i.e., .

Ii-B Problem Statement and A Motivating Example

We are interested in finding deterministic and probabilistic conditions on the sampling pattern tensor under which there are finite completions of the sampled tensor that satisfy .

First we compare the following two approaches in an example to emphasize the necessity of our analysis for general order tensors: (i) analyzing each unfolding individually with the rank constraint of the corresponding unfolding, (ii) analyzing via TT decomposition that incorporates all rank components simultaneously. In particular, we will show via an example that analyzing each of the unfoldings separately is not enough to guarantee finite completability when all rank components are given, while we show that for the same example TT decomposition ensures finite completability.

Consider a -way tensor with TT rank . Assume that four entries of this tensor are observed: and . Observe that the first unfolding is also the first matricization . Moreover, the second unfolding is the transpose of the third matricization since is a three-way tensor. Therefore, the first and second components of the TT rank are the first and the third components of the Tucker rank, respectively.

It is shown in Section II of [27] that having any entries of a rank- matrix, there are infinitely many completions for it. As a result, the first and second unfoldings each is infinitely many completable given only the corresponding rank constraint. Note that the analysis on Grassmannian manifold in [25] is not capable of incorporating more than one rank constraint. However, as we show next the intersection of the mentioned two infinite sets (having both of the rank constraints) is a finite set.

We take advantage of both elements of TT rank simultaneously, in order to show there exist only finitely many completions. Given the TT rank , we define , , (where ) and . Using the decomposition in (1), we have the followings

(4)

Recall that and are the observed entries. Hence, the unknown entries can be determined uniquely in terms of the observed entries as

(5)

Iii Deterministic Conditions for Finite Completability

This section characterizes the connection between the sampling pattern and the number of solutions of a low-rank tensor completion. In Section III-A, we define a polynomial based on each observed entry. Then, given the rank vector, we transform the problem of finite completability of to the problem of including enough number of algebraically independent polynomials among the defined polynomials for the observed entries. In Section III-B, we construct a constraint tensor based on the sampling pattern . This tensor is useful for analyzing the algebraic independency of a subset of polynomials among all defined polynomials. In Section III-C, we show the relationship between the number of algebraically independent polynomials in the mentioned set of polynomials and finite completability of the sampled tensor.

Iii-a Geometry of TT Manifold

Here, we briefly mention some facts to highlight the fundamentals of our proposed analysis. Recall that .

  • Fact : As it can be seen from (2), any observed entry results in an equation that involves entries of , . Considering the entries of as variables (right-hand side of (2)), each observed entry results in a polynomial in terms of these variables.

  • Fact : As it can be seen from (2), for any observed entry , the value of specifies the location of the entries of that are involved in the corresponding polynomial, . In other words, the value of specifies the row number of the second (first) matricization of which its entries are involved in the corresponding polynomial, ().

  • Fact : It can be concluded from Bernstein’s theorem [43] that in a system of polynomials in variables with coefficients chosen generically, the polynomials are algebraically independent with probability one, and therefore there exist only finitely many solutions. Moreover, in a system of polynomials in variables (or less), polynomials are algebraically dependent with probability one.

Given all observed entries , we are interested in finding the number of possible solutions in terms of entries of (infinite or finite) via investigating the algebraic independence among these polynomials.

We are interested in providing a structure on the decomposition such that there is one decomposition among all possible decompositions of the sampled tensor that captures the structure. Before describing such a structure on TT decomposition, we start with a similar structure for matrix decomposition.

Lemma 1.

Let denote a generically chosen matrix from the manifold of matrices of rank . Then, there exists a unique decomposition such that , and , where represents the submatrix of consists of the first columns and the first rows and denotes the identity matrix.

Proof.

We show that there exists exactly one decomposition such that with probability one. Considering the first rows of , we conclude . Therefore, we need to show that there exists exactly one such that or equivalently . It suffices to show that each column of can be determined uniquely having where and . As is a generically chosen matrix of rank , we have with probability one. Hence, results in independent degree- equations in terms of the variables (entries of ), and therefore has exactly one solution with probability one. ∎

Remark 1.

Note that the genericity assumption is necessary as we can find counter examples for Lemma 1 in the absence of genericity assumption, e.g., it is easily verified that the following decomposition is not possible:

           

Remark 2.

Assume that is an arbitrary given full rank matrix. Then, for any submatrix111Specified by a subset of rows and a subset of columns (not necessarily consecutive). of , Lemma 1 also holds if we replace by in the statement. The proof is similar to the proof of Lemma 1 and thus it is omitted.

As mentioned earlier, similar to the matrix case, we are interested in obtaining a structure on TT decomposition of a tensor such that there exists one decomposition among all possible TT decompositions of a tensor that captures the structure. Hence, we define the following structure on the decomposition in order to characterize a condition on the sampling pattern to study the algebraic independency of the above-mentioned polynomials.

Definition 1.

Consider any submatrices of , respectively such that (i) , , (ii) the columns of corresponding to columns of belong to disjoint rows of , . Then, is said to have a proper structure if is full rank, . 222Since and are two-way tensors, i.e., matrices we also denote them by and . Moreover, since is a three-way tensor, ,

Define the following matrices:

(6)

and

(7)

where and . It is easy to verify that satisfy properties (i) and (ii) in Definition 1.

Definition 2.

(Canonical basis) We call a canonical decomposition if for we have , where is the identity matrix.

Lemma 2.

Consider the TT decomposition in (2). Then, , , and , , are full rank matrices.

Proof.

In general, besides the separation rank , we may be able to obtain a TT decomposition for other vectors as well. However, according to [42] among all possible TT decomposition for different values of ’s, , , is minimal, in the sense that there does not exist any decomposition with ’s such that for and for at least one . By contradiction, assume that is not full rank. Then, .

Let denote the matrix . Since , there exists a decomposition such that and also . Hence, the existence of the TT decomposition with and replaced by and contradicts the above-mentioned minimum property of the separation rank. Note that for a three-way tensor, the second unfolding is the transpose of the third matricization, and therefore and the rest of the cases can be verified similarly. ∎

Lemma 3.

Assume that is an arbitrary given full rank matrix, . Consider a set of matrices that satisfy properties (i) and (ii) in Definition 1. Then, there exists exactly one decomposition of the sampled tensor such that , .

Proof.

Consider an arbitrary decomposition of the sampled tensor . Let , , where the above multiplication is the same tensor multiplication in TT decomposition (1). Note that for a three-way tensor, the second unfolding is the transpose of the third matricization, and therefore their ranks are the same. According to Lemma 2, , , , and .

As a result, we have , . Observe that , and therefore for and similarly and . According to Lemma 1 and Remark 2, for an matrix of rank there exists a unique decomposition such that and and an arbitrary submatrix of is equal to the given full rank matrix.

We claim that there exist such that and the corresponding submatrix is equal to the given full rank matrix , . We repeat this procedure for each and update two components of TT decomposition at iteration and at the end, we obtain a TT decomposition that has the mentioned structure in the statement of Lemma 3. In the following we show the existence of such at each iteration. At step one, we find such that and the corresponding submatrix of is equal to . We update the decomposition with and replaced by and , and therefore we obtain a new decomposition of the sampled tensor such that the submatrix of corresponding to is equal to . Then, in step we consider and similarly we update the second and third term of the decomposition obtained in the last step. Eventually after steps, we obtain a decomposition of the sampled tensor that , .To show the uniqueness of such decomposition, we show that each component of the TT decomposition can be determined uniquely. Remark 2 for rank component results that and the multiplication of the rest of the components of the TT decomposition can be determined uniquely. By repeating this procedure for other rank components the uniqueness of such decomposition can be verified by showing the uniqueness of the components one by one. ∎

Lemma 3 leads to the fact that given , the dimension of all tuples that satisfy TT decomposition is , as is the total number of entries of and is the total number of the entries of the pattern or structure that is equivalent to the uniqueness of TT decomposition. We make the following assumption which will be referred to, when it is needed.

Assumption : Each column of includes at least observed entries.

Remark 3.

Note that given , polynomials in (2) are degree- polynomials in terms of the entries of . Hence, Assumption results in degree- polynomials in terms of the entries of . As a result, the entries of can be determined uniquely in terms of the entries of .

Definition 3.

Let denote the set of polynomials corresponding to the observed entries as in (2) excluding the observed entries of Assumption . Note that since is already solved in terms of , each polynomial in is in terms of elements of .

The following lemma provides the necessary and sufficient condition on for finite completability of the sampled tensor .

Lemma 4.

Suppose that Assumption holds. For almost every , there exist only finitely many completions of if and only if there exist algebraically independent polynomials in .

Proof.

The proof is omitted due to the similarity to the proof of Lemma in [27]. The only minor difference is that here the dimension is instead of which is the dimension of the core for Tucker decomposition.

Iii-B Constraint Tensor

In the following, we provide a procedure to construct a binary tensor based on such that and each polynomial can be represented by one -way subtensor of that belongs to . Using , we are able to recognize the observed entries that have been used to obtain the in terms of the entries of , and we can easily verify if two polynomials in are in terms of the same set of variables. Then, in Section III-C, we characterize the relationship between the maximum number of algebraically independent polynomials in and .

For each subtensor of the sampled tensor , let denote the number of sampled entries in . Specifically, consider any subtensor of the tensor . Then, contributes polynomial equations in terms of the entries of among all polynomials in .

The sampled tensor includes subtensors that belong to and let for denote these subtensors. Define a binary valued tensor , where and its entries are described as the following. We can look at as tensors each belongs to . For each of the mentioned tensors in we set the entries corresponding to the observed entries that are used to obtain equal to . For each of the other observed entries, we pick one of the tensors of and set its corresponding entry (the same location as that specific observed entry) equal to and set the rest of the entries equal to . In the case that we simply ignore , i.e.,

By putting together all tensors in dimension , we construct a binary valued tensor , where and call it the constraint tensor. Observe that each subtensor of the constraint tensor that belongs to , i.e., each column of the -th unfolding of , includes exactly nonzero entries. The following example shows this procedure.

Example 1.

Consider an example in which and and . Assume that if and otherwise, where

represents the set of observed entries. Hence, observed entries belong to , observed entries belong to , and observed entries belong to . As a result, , , and .

Also, assume that the entries that we use to obtain given are , , , , and . Hence, , , and , and therefore the constraint tensor belongs to .

Note that (correspond to entries of that have been used to obtain ), and also for the two other observed entries we have (correspond to ) and (correspond to ) and the rest of the entries of are equal to zero. Similarly, and the rest of the entries of are equal to zero.

Then, if and otherwise, where

Note that each subtensor of that belongs to represents one of the polynomials in besides showing the polynomials that have been used to obtain . More specifically, consider a subtensor of that belongs to with nonzero entries. Observe that exactly of them correspond to the observed entries that have been used to obtain . Hence, this subtensor represents a polynomial after replacing entries of by the expressions in terms of entries of , i.e., a polynomial in .

Iii-C Algebraic Independence

In Lemma 4, we obtained the required number of algebraically independent polynomials in for finite completability, and therefore we can certify finite completability based on the maximum number of algebraically independent polynomials in . In this subsection, a sampling pattern on the constraint tensor is proposed to obtain the maximum number of algebraically independent polynomials in based on the structure of the nonzero entries of .

Definition 4.

Let be a subtensor of the constraint tensor . Let denote the number of nonzero rows of . Also, let denote the set of polynomials that correspond to nonzero entries of .

Recall Facts and regarding the number of involved entries of components of the TT decomposition in a set of polynomials. However, according to Lemma 3 some of the entries of ’s are known, i.e., that satisfy properties (i) and (ii) in Definition 1. Therefore, in order to find the number of variables (unknown entries of ’s) in a set of polynomials, we should subtract the number of known entries in the corresponding pattern from the total number of involved entries.

For any subtensor of the constraint tensor, the next lemma gives an upper bound on the number of algebraically independent polynomials in the set