Structured condition numbers for the total least squares problem with linear equality constraint and their statistical estimation

05/17/2021 ∙ by Mahvish Samar, et al. ∙ 0

In this paper, we derive the mixed and componentwise condition numbers for a linear function of the solution to the total least squares with linear equality constraint (TLSE) problem. The explicit expressions of the mixed and componentwise condition numbers by dual techniques under both unstructured and structured componentwise perturbations is considered. With the intermediate result, i.e. we can recover the both unstructured and structured condition number for the TLS problem. We choose the small-sample statistical condition estimation method to estimate both unstructured and structured condition numbers with high reliability. Numerical experiments are provided to illustrate the obtained results.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

We consider the total least squares problem with equality constraint (TLSE):

(1.1)

where denotes the Frobenius norm of a matrix, and the matrix has full row-rank, has full column-rank. By adding some restrictions, this model leads to several distinct problems: for the ordinary total least squares (OTLS) problem [1, 4]; E = 0 for the least squares problem with equality constraint (LSE) [2, 3] and some columns of are zero for the mixed least squares-total least squares problem (MTLS) [[193], 4].

In 1992, E.M. Dowling et al. [5] first investigated the properties of Total least squares with linear equality constraint (TLSE). They proposed to solve it on the basis of QR and SVD matrix factorizations. B. Schaffrin developed an iterative method for solving (1.1) based on the Euler-Lagrange theorem [7]. Recently, Liu et al. [6] interpreted the TLSE solution as an approximation of the solution to an unconstrained weighted TLS problem (WTLS), with a large weight assigned on the constraint, based on which a QR-based inverse iteration method was presented. The superiority of this method over the iteration algorithm in [7] and QR-SVD method [5] was demonstrated by numerical experiments. Condition number plays an important role in perturbation theory and error analysis for algorithms; see e.g., [8, 9, 10]. Recently, the condition numbers of TLS problem, the scaled TLS problem, the multidimensional TLS problem, the mixed LS-TLS problem, truncated-TLS problem and TLSE problem have been considered; see [19, 20, 21, 22, 23, 24, 25, 26, 27]. Structured TLS problems [11, 12, 13] had been studied extensively in the past decades. For structured TLS problems, it is suitable to investigate structured perturbations on the input data, because structure-preserving algorithms that preserve the underlying matrix structure can enhance the accuracy and efficiency of the TLS solution computation. Recently, structured condition numbers of TLS problem, the mixed LS-TLS problem, and truncated-TLS problem have been considered; see [14, 15, 16, 17, 18] However, to our best knowledge, there is no work on structured perturbation analysis and structured condition numbers of TLSE problem so far. Therefore, in this paper, we study structured condition numbers of TLSE problem, including the normwise, mixed, and componentwise condition number and their relationships corresponding to the unstructured ones.

The rest of the paper is organized as follows. Section 2 provides some useful notation and preliminary knowledge. The explicit expressions of relevant unstructured condition numbers are given in Section 3. Structured condition number for the TLSE problem is presented in Section 4. In Section 5, we investigate the statistical estimation of these structured and unstructured condition numbers by small-sample statistical condition estimation (SSCE) method [29]. Section 6 presents some numerical examples to verify the theoretical results.

2 Preliminaries

In this section we first recall some well known results about the TLSE problem. Denote

(2.1)

In [9] the QR-SVD procedure for computing the TLSE solution first factorizes into the QR form:

(2.2)

and then computes the SVD of as

(2.3)

If contains a nonzero last component, the TLSE solution is then uniquely determined by normalizing the last component in to -1 i.e., where up to a factor 1. In [21], Liu et al. carried out further investigations on the uniqueness condition and the explicit closed form for the TLSE solution. For the thin QR factorization of :

(2.4)

let be the minimum 2 -nom solution to and set . Assume that and the specific matrix

(2.5)

have orthonormal columns and the spans of the columns are the null space of and , respectively, then with the genericity condition

(2.6)

the TLSE problem has a unique solution taking the fom

(2.7)

where and with

(2.8)

For the condition number of the TLSE problem, the following first order pertur- bation result is vital.

Lemma 2.1.

Let

(2.9)

Consider the following line a function of the TLSE solution:

(2.10)

where . Then, from [6, Theorem 4.1], we know that With the notationin and the genericity assumption the function is a continious mapping on In addition, is Fréchet differentiable at and its Fréchet derivative is given by

(2.11)

where

Given the perturbations of , of , of , and of . Under the genericity condition (2.2), when is small enough, the perturbed TLSE problem

has a unique TLSE solution . The absolute normwise condition number [1] of a linear function of the TLSE solution, is defined by

where is the TLSE solution of ( 2.7 ) and the last equality is from [30]. Here, we only choose Q. Liu et al. [26, 27] derived the exact expression of as follows

where
The relative normwise condition number corresponding to in (2.8) can be defined by

2.1 Dual techniques

For the Euclidean spaces and equipped with the scalar products and , respectively, we denote the corresponding norms as and , and let a linear operator be well-defined. Thus, the dual norm and the adjoint operator can be defined as follows.

Definition 2.2.

The dual norm of the norm is defined by

where .

Definition 2.3.

The adjoint operator of , is defined by

where and .

For commonly used vector norms and the Frobenius norm, their dual norms are given by

For the linear operator from to , let be the operator norm induced by the norms and , and for the linear operator from to , let be the operator norm induced by the dual norms and . We have the following result [28] on these two norms.

Lemma 2.4.

For the linear operator from to , we have

As noted in [28], it may additionally be desirable to compute in place of when the dimension of the Euclidean space is lower than .

From [8], if is Fréchet differentiable in the neighborhood of , then the absolute condition number of at is given by

(2.12)

where denotes the Fréchet differential of at . In view of Lemma 2.4, the following expression of can be obtained

(2.13)

If is nonzero, we have the relative condition number

(2.14)

Now, we consider the componentwise metric on a data space . For any input data , we denote by the subset of all elements satisfying that whenever . Thus, we can measure the perturbation of using the following componentwise norm with respect to

Equivalently, the componentwise norm has the following property

(2.15)

For this norm, from [15, Equation 2.16], we have the following explicit expression of its dual norm

(2.16)

Using the above componentwise norm, we can rewrite the condition number .

Lemma 2.5.

[28, 15] Using the above notations and the componentwise norm defined in (2.15), the condition number can be expressed by

(2.17)

where is given by (2.16).

3 Unstructured condition number expressions of TLSE via dual techniques

Lemma 3.1.

The adjoint of operator of the Fréchet derivative in (2.8) is given by

Proof: Using (2,8) and the definition of the scalar product in the matrix space, for any , we have

For the second part of the adjoint of the derivative we have

Let

then

which completes the proof. Now, we give an explicit expression of the condition number for the linear function of the TLSE solution.

Theorem 3.2.

The condition number (9) for the linear function of the TLSE solution is expressed by

where

and denotes the diagonal matrix for any matrix .

Proof: Let and be the entries of and , respectively. Thus, using (12), we have

By Lemma we derive that

where is the ith component of . Noting (18) , it can be verified that is the the column of the matrix Thus, the above expression equals

Then, by Lemma 2.5 , we obtain the desired result. By Theorem 3.2 , we can find the explicit expression of the mixed condition number for the linear function of the TLSE solution easily.

Corollary 3.3.

Using the above notations, when the infinity norm is chosen as the norm in the solution space . we obtain

When the infinity norm is chosen as the nonn in the solution space , the corresponding mixed condition number is given by

(3.1)
Corollary 3.4.

Considering the componentwise norm defined by

in the solution space, we have the following three expressions for the componentwise condition number for the linear function of of the TLSE solution

4 Structured condition number expressions of TLSE via dual techniques

Assume that is linear structured, i.e., , where form a basis of and . In view of and , where , is column orthogonal and has full column rank, with at most one nonzero entry in each row. Note that

and for the perturbed TLSE problem, if we restrict the perturbation matrices to have the same structure as that of , that is, where . Define the mapping from to such that . Based on , the first order perturbation result becomes . According to the concept of condition numbers, the relative norwise condition numbers for structured TLSE has following forms

where

which is Kronecker product-free and can be computable more efficiently with less storage.

From Lemma we can prove defined by (1.4) is Fréchet differentiable at and derive its Fréchet derivative in the follow lemma.

Lemma 4.1.

Consider the following linear function of the TLSE solution:

where . Then, from [6, Theorem 4.1], we now that With the notation in (2.3) and the genericity assumption the function is a continuous mapping on In addition, is Fréchet differentiable at and its Fréchet derivative is given by

Lemma 4.2.

The adjoint of operator of the Fréchet derivative in (2.8) is given by

Theorem 4.3.

The condition number (9) for the linear function of the TLSE solution is expressed by

where

and denotes the diagonal matrix for any matrix . By Theorem we can find the explicit expression of the mixed condition number for the linear function of the TLSE solution easily.

Corollary 4.4.

Using the above notations, when the infinity norm is chosen as the norm in the solution space , we obtain

When the infinity norm is chosen as the norm in the solution space , the corresponding mixed condition number is given by

(4.1)
Corollary 4.5.

Considering the componentwise norm defined by (3.5 )in the solution space, we have the following two expressions for the componentwise condition number for the linear function of the TLSE solution

(4.2)
Theorem 4.6.

Suppose that the basis for satisfies for any in then

Proof. Using the monotonicity of the infinity norm, we have

(4.3)

for the last equality we use the assumption . With the above inequality, and the expressions of , it is easy to prove the first two inequalities in this theorem.

Remark 4.7.

Here, we can recover the structured mixed and componentwise condition numbers for the TLS problem [15], by setting and in (2.9) . In this case,

where , and by setting and in (2.9) and from which and

with being a Householder matrix, therefore the estimate in Theorem 4 becomes

5 Statistical condition estimates

In this section, we introduce three algorithms for estimating the normwise, mixed and componentwise condition numbers for the TLSE Problem.

5.1 Estimating unstructured normwise, mixed and componentwise condition numbers

We use two algorithms to estimate the unstructured normwise, mixed and componentwise condition number. The first one, outlined in Algorithm 1, is based on the SSCE (small-sample statistical condition estimation) method [29] and has been applied to estimate the unstructured normwise condition number (see [31, 32, 33, 16, 17]). The second one, outlined in Algorithm 2, is also from [29] and has been used to estimate the unstructured mixed and componentwise condition numbers.

  1. Generate matrices with entries being in

    standard Gaussian distribution. Orthonormalize the following matrix

    to obtain via the modified Gram-Schmidt orthogonalization process. Each can be converted into the desired matrices with the unvec operation.

  2. Let . Approximate and by

    (5.1)
  3. For compute

    (5.2)
  4. Estimate the absolute condition vector

    where the power operation is applied at each entry of and with

  5. Compute the normwise condition number as follows,

    where

Algorithm 1 SSCE method for the unstructured normwise condition number for the TLSE problem
  1. Generate matrices with entries being in . Orthonormalize the following matrix

    to obtain via modified Gram-Schmidt orthogonalization process. Each can be converted into the desired matrices with the unvec operation. Let be the matrix multiplied by componentwise.

  2. Let . Approximate and by (5.1).

  3. For compute

    Using the approximations for and estimate the absolute condition vector