In this paper, we consider the following linear model
The TLS problem is often used for the linear model (1.1) when the augmented matrix is rank deficient, i.e., the small singular values of are assumed to be separated from the others. More interestingly, the truncated total least square (TTLS) method aims to solve the linear model (1.1) in the sense that the small singular values of are set to be zeros. For the discussion of the TTLS, one may refer to [16, §3.6.1] and [6, 7]
. The TTLS problem arises in various applications such as linear system theory, computer vision, image reconstruction, system identification, speech and audio processing, modal and spectral analysis, and astronomy, etc. The overview of the TTLS can be found in.
Let be the predefined truncated level, where . The TTLS problem aims to solve the following problem:
where denotes the Euclidean vector norm or its induced matrix norm and is the best rank- approximation of in the Frobenius norm.
In order to solve (1.3), we first recall the singular value decomposition (SVD) of which is given by
where and are orthogonal matrices and is a real matrix with a vector on its diagonal and , where . Here, is a diagonal matrix with a vector on its diagonal and the superscript “” takes the transpose of a matrix or vector. Then we have , where . Suppose the truncation level satisfies the condition
then the TTLS problem is generic and the TTLS solution is given by 
Condition numbers measure the worst-case sensitivity of an input data to small perturbations. Normwise condition numbers for the TLS problem (1.2) under the genericity condition were studied in [1, 17], where SVD-base explicit formulas for normwise condition numbers were derived. The normwise condition number of the truncated SVD solution to a linear model as (1.1) was introduced in . When the data is sparse or badly scaled, it is more suitable to consider the componentwise perturbations since normwise perturbations only measure the perturbation for the data by means of norm and may ignore the relative size of the perturbation on its small (or zero) entries (cf. ). There are two types of condition numbers in componentwise perturbations. The mixed condition number measures the errors in the output using norms and the input perturbations componentwise, while the componentwise condition number measures both the error in the output and the perturbation in the input componentwise (cf. ). The Kronecker product based formulas for the mixed and componentwise condition numbers to the TLS problem (1.2) were derived in [30, 4]. The corresponding componentwise perturbation analysis for the multidimensional TLS problem and mixed least squares-TLS problem can be found in [28, 29].
Gratton et al. in  investigated the normwise condition number for the TTLS problem (1.3). The normwise condition number formula and its easier computable upper bounds for the TTLS solution (1.3) were derived (cf. [14, Theorems 2.4-2.6]), which rely on the SVD of the augmented matrix . Since the normwise condition number formula for the TTLS problem (1.3) involves Kronecker product, which is not easy to compute or evaluate even for the medium size TTLS problem, the condition estimation method based on the power method  or the Golub-Kahan-Lanczos (GKL) bidiagonalization algorithm  was proposed to estimate the spectral norm of Fréchet derivative matrix related to (1.3).
As mentioned before, when the TTLS problem (1.3) is sparse or badly scaled, which often occurs in scientific computing, the conditioning based on normwise perturbation analysis may severely overestimate the exact error of the numerical solution to (1.3). Indeed, from the numerical results for Example 5.1 in Section 5, the TTLS problem (1.3) with respect to the specific data and is well-conditioned under componentwise perturbation analysis while it is very ill-conditioned under normwise perturbation, which implies that the normwise relative errors for the numerical solution to (1.3) are pessimistic. In this paper, we propose the mixed and componentwise condition number for the TTLS problem (1.3) and the corresponding explicit expressions are derived, which can capture the exact conditioning of (1.3) with respect to the sparsity and scaling for the input data. Furthermore, when the truncated level in (1.3) is selected to be , (1.3) reduces to (1.2). The normwised, mixed and componentwise condition numbers for the TTLS problem (1.3) are shown to be mathematically equivalent to the corresponding ones [1, 17, 30] for the untruncated case from their explicit expressions.
Structured TLS problems [18, 23, 26] had been studied extensively in the past decades. For structured TLS problems, it is suitable to investigate structured perturbations on the input data, because structure-preserving algorithms that preserve the underlying matrix structure can enhance the accuracy and efficiency of the TLS solution computation. Structured condition numbers for structured TLS problems can be found in [24, 4, 5] and references therein. In this paper, we introduce structured perturbation analysis for the structured TTLS (STTLS) problem. The explicit structured normwise, mixed and componentwise condition numbers for the STTLS problem are obtained, and their relationships corresponding to the unstructured ones are investigated.
The Kronecker product based expressions for both unstructured and structured normwise, mixed and componentwise condition numbers of the TTLS solution involve higher dimensions thus prevent the efficient calculations of these condition numbers. In practice, it is important to estimate condition numbers efficiently since the forward error for the numerical solution can be obtained via combining condition numbers with backward errors. In this paper, based on the small sample statistical condition estimation (SCE) , we propose reliable condition estimation algorithms for both unstructured and structured normwise, mixed and componentwise condition numbers of the TTLS solution, which utilize the SVD of to reduce the computational cost. Furthermore, the proposed condition estimation algorithms can be integrated into the SVD-based direct solver for the small or medium size TTLS problem (1.3). Therefore, one can obtain the reliable forward error estimations for the numerical TTLS solution after implementing the proposed condition estimation algorithms. The main computational cost in condition number estimations for (1.3) is to evaluate the directional derivatives with respect to the generated direction during the loops in condition number estimations algorithms. We point out that the power method  for estimating the normwise condition number in  needs to evaluate the directional derivatives twice in one loop. However, only evaluating direction derivative once is needed in the loop of Algorithms 1 to 3. Therefore, compared with the normwise condition number estimation algorithm proposed in , our proposed condition number estimations algorithms in this paper are more efficient in terms of the computational complexity, which are also applicable for estimating the componentwise and structured perturbations for (1.3). For recent SCE’s developments for (structured) linear systems, linear least squares and TLS problem, we refer to [21, 20, 22, 5] and references therein.
The rest of this paper is organized as follows. In Section 2 we review pervious perturbation results on the TTLS problem and derive explicit expressions of the mixed and componentwise condition numbers. The structured normwise, mixed and componentwise condition numbers are also investigated in Section 2, where the relationships between the unstructured normwise, mixed and componentwise condition numbers for (1.3) with the corresponding structured counterparts are investigated. In Section 4 we are devoted to propose several condition estimation algorithms for the normwise, mixed and componentwise condition numbers of the TTLS problem. Moreover, the structured condition estimation is considered. In Section 5, numerical examples are shown to illustrate the efficiency and reliability of the proposed algorithms and report the perturbation bounds based on the proposed condition number. Finally, some concluding remarks are drawn in the last section.
2. Condition numbers for the TTLS problem
In this section we review previous perturbation results on the TTLS problem. The explicit expressions of the mixed and componentwise condition numbers for the TTLS problem are derived. Furthermore, for the structured TTLS problem, we propose the normwise, mixed and componentwise condition numbers, where explicit formulas for the corresponding counterparts are obtain. The relationships between the unstructured normwise, mixed and componentwise condition numbers for (1.3) with the corresponding structured counterparts are investigated. We first introduce some conventional notations in the subsequent paper.
Throughout this paper, we use the following notation. Let be the vector -norm or its induced matrix norm. Let
be the identity matrix of order. Let be the -th column vector of an identity matrix of an appropriate dimension. The superscript “” means the Moore-Penrose inverse of a matrix. The symbol “” means componentwise multiplication of two conformal dimensional matrices. For any matrix , let , where denote the absolute value of . For any two matrices , represents for all and . For any , we define by
Let be a column vector obtained by stacking the columns of on top of one another. For a vector , let , where for and . The symbol “” means the Kronecker product and is a permutation matrix defined by
Given the matrices , , and , and with appropriate dimensions, we have the following propertes of the Kronecker product and vec operator :
where the function is given by
Please be noted the the dimension of may be large even for medium size TTLS problems. The explicit formula given by (2.5) involves the computation of the spectral norm of . Hence, upper bounds for is obtain in [14, §2.4], which only rely on the singular values of and . When the data is sparse or badly scaled, the normwise condition number may not reveal the conditioning of (1.3), since normwise perturbations ignore the relative size of the perturbation on its small (or zero) entries. Therefore, it is more suitable to consider the componentwise perturbation analysis for (1.3). In the next subsection, we shall introduce the mixed and componentwise condition number for (1.3).
In [14, §2.3], if both and the full SVD of are available, then one may compute by using the power method [15, Chap. 15] to or the Golub-Kahan-Lanczos (GKL) bidiagonalization algorithm  to or , where only the matrix-vector product is needed. However, as pointed in the introduction part, the normwise condition number estimation algorithm in  are devised based on the power method , which needs to evaluate and in one loop for some suitable dimensional vectors and . In Section 4, SCE-based condition estimation algorithms for (1.3) shall be proposed, where in one loop we only need to compute the directional derivative and is not involved. Therefore, compared with normwise condition number estimation algorithm in , SCE-based condition estimation algorithms are more efficient.
2.2. Mixed and componentwise condition numbers
In Lemma 2.1 below, the first order perturbation expansion of with respect to the perturbations of the data and is reviewed, which involves the Kronecker product. In order to avoid the Kronecker product in the explicit expression for the directional derivative of , we derive the corresponding equivalent formula (2.7) in Lemma 2.2. Furthermore, (2.7) can be used to save computation memory of SCE-base condition estimation algorithms in Section 4.
Under the same assumptions as in Lemma 2.1, if with sufficiently small, then the directional derivative of at in the direction is given by
Proof. From Lemma 2.1 we have
Using the fact that and (2.2) we have
From (2.6), we see that the -th diagonal block of is given by
for . By the definition of we have
Then, using the partition of given by (1.6) we have
This completes the proof. ∎
When the data is sparse or badly-scaled, it is more suitable to adopt the componentwise perturbation analysis to investigate the conditioning of the TTLS problem. In the following definition, we introduce the relative mixed and componentwise condition number for the TTLS problem.
Suppose the truncation level is chosen such that and . The mixed and componentwise condition numbers for the TTLS problem (1.3) are defined as follows:
In the following theorem, we give the explicit expressions of and .
Proof. Let , where and . For any , it follows from that
By Lemma 2.1 we have for sufficiently small,
and taking infinity norms we have
where the fact that is used. Thus,
On the other hand, let the index is such that
where denotes the -th row of . We choose
where is a diagonal matrix such that =sign for . Using (2.2) we have
Using the fact that
it is easy to see that
From Example 5.1, we can see can be much smaller than when the data is sparse and badly scaled. Therefore, one should adopt the mixed and componentwise condition number to measure the conditioning of (1.3) instead of the normwise condition number when is spare or badly scaled. Furthermore, in practice it is necessary to propose efficient and reliable condition estimations for and , which will be investigated in Section 4.
In [3, 18, 23], the structured TLS (STTLS) problem has been studied extensivley. Hence, it is interesting to study the structured perturbation analysis for the STTLS problem. In the following, we propose the structured normwise, mixed and componentwise condition numbers for the STTLS problem, where is a linear structured data matrix. Assume that is a linear subspace which consists of a class of basis matrices. Suppose there are () linearly independent matrices in , where are matrices of constants, typically 0s and 1s. For any , there is a uniques vector such that
In the following, we study the sensitivity of the STTLS solution to perturbations on the data and , which is defined by
In the following lemma, we provide the first order expansion of the STTLS solution with respect to the structured perturbations on and on , which help us to derive the structured condition number expressions for the STTLS problem (1.3) and (2.17). In view of the fact that , we can prove the following lemma from Lemma 2.1. The detailed proof is omitted here.
Under the same assumptions of Lemma 2.1, if with sufficiently small, then, for the STTLS solution of and the STTLS solution of , we have
The following theorem concerns with the explicit expressions for the structured normwise, mixed and componentwise condition numbers , , and defined in Definition 2.2 when can be expressed by (2.17). Since the proof is similar to Theorem 2.1, we omit it here.