 # Structured condition numbers for the total least squares problem with linear equality constraint and their statistical estimation

In this paper, we derive the mixed and componentwise condition numbers for a linear function of the solution to the total least squares with linear equality constraint (TLSE) problem. The explicit expressions of the mixed and componentwise condition numbers by dual techniques under both unstructured and structured componentwise perturbations is considered. With the intermediate result, i.e. we can recover the both unstructured and structured condition number for the TLS problem. We choose the small-sample statistical condition estimation method to estimate both unstructured and structured condition numbers with high reliability. Numerical experiments are provided to illustrate the obtained results.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

We consider the total least squares problem with equality constraint (TLSE):

 minE,f∥[E,f]∥F, subject to (A+E)x=b+f,Cx=d (1.1)

where denotes the Frobenius norm of a matrix, and the matrix has full row-rank, has full column-rank. By adding some restrictions, this model leads to several distinct problems: for the ordinary total least squares (OTLS) problem [1, 4]; E = 0 for the least squares problem with equality constraint (LSE) [2, 3] and some columns of are zero for the mixed least squares-total least squares problem (MTLS) [, 4].

In 1992, E.M. Dowling et al.  first investigated the properties of Total least squares with linear equality constraint (TLSE). They proposed to solve it on the basis of QR and SVD matrix factorizations. B. Schaffrin developed an iterative method for solving (1.1) based on the Euler-Lagrange theorem . Recently, Liu et al.  interpreted the TLSE solution as an approximation of the solution to an unconstrained weighted TLS problem (WTLS), with a large weight assigned on the constraint, based on which a QR-based inverse iteration method was presented. The superiority of this method over the iteration algorithm in  and QR-SVD method  was demonstrated by numerical experiments. Condition number plays an important role in perturbation theory and error analysis for algorithms; see e.g., [8, 9, 10]. Recently, the condition numbers of TLS problem, the scaled TLS problem, the multidimensional TLS problem, the mixed LS-TLS problem, truncated-TLS problem and TLSE problem have been considered; see [19, 20, 21, 22, 23, 24, 25, 26, 27]. Structured TLS problems [11, 12, 13] had been studied extensively in the past decades. For structured TLS problems, it is suitable to investigate structured perturbations on the input data, because structure-preserving algorithms that preserve the underlying matrix structure can enhance the accuracy and efficiency of the TLS solution computation. Recently, structured condition numbers of TLS problem, the mixed LS-TLS problem, and truncated-TLS problem have been considered; see [14, 15, 16, 17, 18] However, to our best knowledge, there is no work on structured perturbation analysis and structured condition numbers of TLSE problem so far. Therefore, in this paper, we study structured condition numbers of TLSE problem, including the normwise, mixed, and componentwise condition number and their relationships corresponding to the unstructured ones.

The rest of the paper is organized as follows. Section 2 provides some useful notation and preliminary knowledge. The explicit expressions of relevant unstructured condition numbers are given in Section 3. Structured condition number for the TLSE problem is presented in Section 4. In Section 5, we investigate the statistical estimation of these structured and unstructured condition numbers by small-sample statistical condition estimation (SSCE) method . Section 6 presents some numerical examples to verify the theoretical results.

## 2 Preliminaries

In this section we first recall some well known results about the TLSE problem. Denote

 ˜A=[A,b],˜E=[E,f],˜C=[C,d] (2.1)

In  the QR-SVD procedure for computing the TLSE solution first factorizes into the QR form:

 ˜CT=˜Q˜R=[˜Q1,˜Q2][˜R10]=˜Q1˜R1, with ˜Q1∈R(n+1)×p,˜R1∈Rp×p (2.2)

and then computes the SVD of as

 ˜A˜Q2=˜U˜Σ˜VT=n−p+1∑i=1~σi~uivTi (2.3)

If contains a nonzero last component, the TLSE solution is then uniquely determined by normalizing the last component in to -1 i.e., where up to a factor 1. In , Liu et al. carried out further investigations on the uniqueness condition and the explicit closed form for the TLSE solution. For the thin QR factorization of :

 CT=Q[R10]=[Q1,Q2][R10]=Q1R1 (2.4)

let be the minimum 2 -nom solution to and set . Assume that and the specific matrix

 ~Q2=[Q2ζxC0−ζ] with ζ=(1+∥xC∥22)−1/2 (2.5)

have orthonormal columns and the spans of the columns are the null space of and , respectively, then with the genericity condition

 σn−p(AQ2)=¯σn−p>~σn−p+1:=σn−p+1(AQ2,ζrC) (2.6)

the TLSE problem has a unique solution taking the fom

where and with

 H=(B(ATA−~σ2n−p+1In)B)†,C†A=(In−HATA)C† (2.8)

For the condition number of the TLSE problem, the following first order pertur- bation result is vital.

###### Lemma 2.1.

Let

 K=[CA],h=[db] (2.9)

Consider the following line a function of the TLSE solution:

 (2.10)

where . Then, from [6, Theorem 4.1], we know that With the notationin and the genericity assumption the function is a continious mapping on In addition, is Fréchet differentiable at and its Fréchet derivative is given by

 J:=dϕ(K,h)⋅(dK,dh)=L(2ρ−2HxtT−[C†A,HAT])(dKx−dh)−LHdKTt (2.11)

where

Given the perturbations of , of , of , and of . Under the genericity condition (2.2), when is small enough, the perturbed TLSE problem

 minE,f∥[E,f]∥F,subject to((A+ΔA)+E)x=(b+Δb)+f,(C+ΔC)x+Δx=d+Δd

has a unique TLSE solution . The absolute normwise condition number  of a linear function of the TLSE solution, is defined by

 κ(L,K,h) =limϵ→0sup∥[ΔK,Δh]∥F≤ϵ∥LΔx∥2ϵ =max[ΔK,Δh]≠0∥Ldϕ(K,h)⋅(ΔK,Δh)∥2∥[ΔK,Δh]∥F,

where is the TLSE solution of ( 2.7 ) and the last equality is from . Here, we only choose Q. Liu et al. [26, 27] derived the exact expression of as follows

 κ(L,K,h)=∥QK,h∥2

where
The relative normwise condition number corresponding to in (2.8) can be defined by

 κn(L,K,h)=limϵ→0sup∥[ΔK,Δh]∥F≤ϵ∥[K,h]∥F∥LΔx∥2∥Lx∥2=κ(L,K,h)∥[K,h]∥F∥Lx∥2

### 2.1 Dual techniques

For the Euclidean spaces and equipped with the scalar products and , respectively, we denote the corresponding norms as and , and let a linear operator be well-defined. Thus, the dual norm and the adjoint operator can be defined as follows.

###### Definition 2.2.

The dual norm of the norm is defined by

 ∥a∥S∗=maxw≠0⟨a,w⟩S∥w∥S,

where .

###### Definition 2.3.

The adjoint operator of , is defined by

 ⟨y,Ma⟩Q=⟨M∗y,a⟩S,

where and .

For commonly used vector norms and the Frobenius norm, their dual norms are given by

 ∥⋅∥1∗=∥⋅∥∞, ∥⋅∥∞∗=∥⋅∥1, ∥⋅∥2∗=∥⋅∥2,∥A∥F∗=∥A∥F.

For the linear operator from to , let be the operator norm induced by the norms and , and for the linear operator from to , let be the operator norm induced by the dual norms and . We have the following result  on these two norms.

###### Lemma 2.4.

For the linear operator from to , we have

 ∥M∥S,Q=∥M∗∥Q∗,S∗.

As noted in , it may additionally be desirable to compute in place of when the dimension of the Euclidean space is lower than .

From , if is Fréchet differentiable in the neighborhood of , then the absolute condition number of at is given by

 κ=∥dϕ(a)∥S,Q=max∥z∥S=1∥dϕ(a)⋅z∥Q, (2.12)

where denotes the Fréchet differential of at . In view of Lemma 2.4, the following expression of can be obtained

 κ=max∥da∥S=1∥dϕ(a)⋅da∥Q=max∥z∥Q∗=1∥dϕ(a)∗⋅z∥S∗. (2.13)

If is nonzero, we have the relative condition number

 κn=κ∥a∥S∥ϕ(a)∥Q. (2.14)

Now, we consider the componentwise metric on a data space . For any input data , we denote by the subset of all elements satisfying that whenever . Thus, we can measure the perturbation of using the following componentwise norm with respect to

 ∥da∥c=min{w,|dai|≤w|ai|, i=1,⋯,n}.

Equivalently, the componentwise norm has the following property

 ∥da∥c=max{|dai||ai|, ai≠0}=∥∥ ∥∥(|dai||ai|)∥∥ ∥∥∞. (2.15)

For this norm, from [15, Equation 2.16], we have the following explicit expression of its dual norm

 ∥d(a)∥c∗=∥(∥da1∥S∗,⋯,∥dan∥S∗)∥∞=∥(|da1||a1|,⋯,|dan||an|)∥1. (2.16)

Using the above componentwise norm, we can rewrite the condition number .

###### Lemma 2.5.

[28, 15] Using the above notations and the componentwise norm defined in (2.15), the condition number can be expressed by

 κ=max∥z∥Q∗=1∥(dϕ(a))∗⋅z∥c∗, (2.17)

where is given by (2.16).

## 3 Unstructured condition number expressions of TLSE via dual techniques

###### Lemma 3.1.

The adjoint of operator of the Fréchet derivative in (2.8) is given by

Proof: Using (2,8) and the definition of the scalar product in the matrix space, for any , we have

 ⟨u,J1(u)⟩ =trace(xu⊤L(2ρ−2HxtT−[C†A,HAT])dK)−trace(tu⊤LH(dK)⊤) =⟨[2ρ−2HxtT−[C†A,HAT]]⊤L⊤ta⊤−tu⊤LH,dK⟩

For the second part of the adjoint of the derivative we have

 ⟨u,J2(u)⟩=u⊤L[2ρ−2HxtT−[C†A,HKT]]dh=⟨[2ρ−2HxtT−[C†A,HAT]]⊤L⊤u, dh⟩

Let

 J∗1(u) =[2ρ−2HxtT−[C†A,HAT]]⊤L⊤ux⊤−tu⊤LH J∗2(u) =[2ρ−2HxtT−[C†A,HAT]]⊤L⊤u

then

 ⟨J∗(u),(dK, dh)⟩=⟨(J∗1(u),J∗2(u))),(dK, dh)⟩=⟨u,J(dK, dh)⟩

which completes the proof. Now, we give an explicit expression of the condition number for the linear function of the TLSE solution.

###### Theorem 3.2.

The condition number (9) for the linear function of the TLSE solution is expressed by

where

 V=xT⊗(2ρ−2KxtT−[C†A,KAT])−K⊗tT,S=(2ρ−2KxtT−[C†A,KAT])

and denotes the diagonal matrix for any matrix .

Proof: Let and be the entries of and , respectively. Thus, using (12), we have

 ∥(dK, dh)∥c∗=∑i,j∣∣ dkij∥kij∣∣+∑i∣∣ dhi∣∣|hi|

By Lemma we derive that

 ∥J∗(u)∥c∗= m,n∑i,j=1∣∣kij∣∣∣∣∣([2ρ−2HxtT−[C†A,HAT]]⊤L⊤ux⊤−tu⊤LH)ij∣∣∣ +m∑i=1|hi|∣∣∣([2ρ−2HxtT−[C†A,HAT]]⊤L⊤u)i∣∣∣ = m,n∑i,j=1∣∣kij∣∣∣∣∣[xj(2ρ−2HxtT−[C†A,HAT])ei−tiHTej]⊤LTu∣∣∣ +m∑i=1|hi|∣∣((2ρ−2HxtT−[C†A,HAT])ei)LTu∣∣

where is the ith component of . Noting (18) , it can be verified that is the the column of the matrix Thus, the above expression equals

 ∥∥ ∥∥[DKVTLuDhSTLu]∥∥ ∥∥1=∥∥[VDK,SDh]TLTu∥∥1

Then, by Lemma 2.5 , we obtain the desired result. By Theorem 3.2 , we can find the explicit expression of the mixed condition number for the linear function of the TLSE solution easily.

###### Corollary 3.3.

Using the above notations, when the infinity norm is chosen as the norm in the solution space . we obtain

 κ∞=∥|LV|vec(|K|)+|LS||h|∥∞

When the infinity norm is chosen as the nonn in the solution space , the corresponding mixed condition number is given by

 κm=∥|LV|vec(|K|)+|LS||h|∥∞∥Lx∥∞ =∥∥∥∣∣∣L[xT⊗(2ρ−2KxtT−[C†A,KAT])−K⊗tT]∣∣∣vec(|K|)+∣∣L[2ρ−2KxtT−[C†A,KAT]]∣∣|h|∥∥∥∞∥Lx∥∞ (3.1)
###### Corollary 3.4.

Considering the componentwise norm defined by

 ∥y∥c=min{w,|yi|≤w|(Lx)i|,i=1,⋯,k}=max{|yi|/|(Lx)1|,i=1,⋯,k}

in the solution space, we have the following three expressions for the componentwise condition number for the linear function of of the TLSE solution

 κc =∥D−1LxL[VDK,SDb]∥∞ =∥∥|D−1Lx|(|LV|vec(|K|)+|LS||h|)∥∥∞

## 4 Structured condition number expressions of TLSE via dual techniques

Assume that is linear structured, i.e., , where form a basis of and . In view of and , where , is column orthogonal and has full column rank, with at most one nonzero entry in each row. Note that

 vec([K,h])=ΦsK,hs:=[Φs K00Im][kh]

and for the perturbed TLSE problem, if we restrict the perturbation matrices to have the same structure as that of , that is, where . Define the mapping from to such that . Based on , the first order perturbation result becomes . According to the concept of condition numbers, the relative norwise condition numbers for structured TLSE has following forms

 κs,n(k,h)=∥∥QΦsK,h∥∥2∥∥[kT,hT]∥∥2∥xTLSE∥2

where

 =([2ρ−2HxtT−[C†A,HAT]][S1x,⋯,Sqx,−Im]−H[ST1r,⋯,STqr,0n×1]).

which is Kronecker product-free and can be computable more efficiently with less storage.

From Lemma we can prove defined by (1.4) is Fréchet differentiable at and derive its Fréchet derivative in the follow lemma.

###### Lemma 4.1.

Consider the following linear function of the TLSE solution:

where . Then, from [6, Theorem 4.1], we now that With the notation in (2.3) and the genericity assumption the function is a continuous mapping on In addition, is Fréchet differentiable at and its Fréchet derivative is given by

###### Lemma 4.2.

The adjoint of operator of the Fréchet derivative in (2.8) is given by

###### Theorem 4.3.

The condition number (9) for the linear function of the TLSE solution is expressed by

 κs=∥∥[VsDK,SDh]TLT∥∥Q,1

where

 Vs=U

and denotes the diagonal matrix for any matrix . By Theorem we can find the explicit expression of the mixed condition number for the linear function of the TLSE solution easily.

###### Corollary 4.4.

Using the above notations, when the infinity norm is chosen as the norm in the solution space , we obtain

 κs,∞=∥|LVs||k|+|LS||h|∥∞

When the infinity norm is chosen as the norm in the solution space , the corresponding mixed condition number is given by

 κs,m =∥|LVs||k|+|LS||h|∥∞∥Lx∥∞ (4.1)
###### Corollary 4.5.

Considering the componentwise norm defined by (3.5 )in the solution space, we have the following two expressions for the componentwise condition number for the linear function of the TLSE solution

 κs,c =∥∥|D−1Lx|(|LVs||k|+|LS||h|)∥∥∞ =∥∥∥|D−1Lx|(q∑i=1|ki|∣∣L((2ρ−2HxtT−[C†A,HAT])Six−HSTit)∣∣ +∣∣L(2ρ−2HxtT−[C†A,HAT])∣∣|h|)∥∥∥ (4.2)
###### Theorem 4.6.

Suppose that the basis for satisfies for any in then

 κs,m≤κmandκs,c≤κc

Proof. Using the monotonicity of the infinity norm, we have

 κs,m ≤∥∥|LV|∑qi=1|ki||vec(Si)|+|LS||h|∥∥∞∥Lx∥∞=∥|LV|vec(|K|)+|LS||h|∥∞∥Lx∥∞ (4.3)

for the last equality we use the assumption . With the above inequality, and the expressions of , it is easy to prove the first two inequalities in this theorem.

###### Remark 4.7.

Here, we can recover the structured mixed and componentwise condition numbers for the TLS problem , by setting and in (2.9) . In this case,

 C†A=0n×p,H=(ATA−σ2n+1In)−1=:¯P−1,tT=[01×p,rT]

where , and by setting and in (2.9) and from which and

 QA,b=(¯P−1[0n×p−ATH0][xT,−1]⊗Im−¯P−1[In,0n×1]⊗rT).

with being a Householder matrix, therefore the estimate in Theorem 4 becomes

 Δx =Q(A,b)vec([ΔA,Δb])+O(∥[ΔA,Δb]∥2F) ≈−(ATA−σ2n+1In)−1ATH0(ΔAx−Δb)+(ATA−σ2n+1In)−1ΔATr

## 5 Statistical condition estimates

In this section, we introduce three algorithms for estimating the normwise, mixed and componentwise condition numbers for the TLSE Problem.

### 5.1 Estimating unstructured normwise, mixed and componentwise condition numbers

We use two algorithms to estimate the unstructured normwise, mixed and componentwise condition number. The first one, outlined in Algorithm 1, is based on the SSCE (small-sample statistical condition estimation) method  and has been applied to estimate the unstructured normwise condition number (see [31, 32, 33, 16, 17]). The second one, outlined in Algorithm 2, is also from  and has been used to estimate the unstructured mixed and componentwise condition numbers.