Singular Vectors From Singular Values

1 Introduction

In the recent paper [1], Denton et al. provided the eigenvector-eigenvalue identity for Hermitian matrices, and a survey was also given for such identity in the literature. The main aim of this paper is to present the identity related to singular vectors and singular values of a general matrix. Indeed, this topic has been studied by Thompson [2]. In particular, the interlacing inequalities for singular values of submstrices are derived in [2].

Let $A$ be a $m \times n$ matrix with singular values $σ_{i} (A)$ and normed left singular vectors $u_{i}$ and right singular vectors $v_{l}$ . $A^{H}$ denotes the conjugate transpose of $A$ . The elements of each left singular vector are denoted $u_{i j}$ and right singular vector $v_{l s}$ . Let ${¯ A}_{j}$ be $(n - 1) \times n$ matrix after deleting the j-th row of $A$ and $σ_{k} ({¯ A}_{j})$ be singular values of ${¯ A}_{j}$ . Let ${^A}_{s}$ be $n \times (n - 1)$ matrix after deleting the s-th column of $A$ and $σ_{t} ({^A}_{s})$ be singular values of ${^A}_{s}$ .

Lemma 1[1] Let $A$ be a $n \times n$ Hermitian matrix with distinct eigenvalues $λ_{i} (A)$ and normed eigenvectors $v_{i} .$ The elements of each eigenvector are denoted $v_{i, j}$ . Let $M_{j}$ be the $(n - 1) \times (n - 1)$ submatrix of $A$ that results from deleting the j-th column and the j-th row, with eigenvalues $λ_{k} (M_{j})$ . Then

| v_{i j} |^{2} = \frac{n - 1 \prod k = 1 (λ_{i} (A) - λ_{k} (M_{j}))}{n \prod k = 1, k \neq i (λ_{i} (A) - λ_{k} (A))}, 1 \leq i, j \leq n .

Lemma 2[1] Let $A$ be a $n \times n$ Hermitian matrix with eigenvalues $λ_{i} (A)$ and normed eigenvectors $v_{i} .$ The elements of each eigenvector are denoted $v_{i, j}$ . Let $M_{j}$ be the $(n - 1) \times (n - 1)$ submatrix of $A$ that results from deleting the j-th column and the j-th row, with eigenvalues $λ_{k} (M_{j})$ . Then

| v_{i j} |^{2} n \prod k = 1, k \neq i (λ_{i} (A) - λ_{k} (A)) = n - 1 \prod k = 1 (λ_{i} (A) - λ_{k} (M_{j})), 1 \leq i, j \leq n .

In this paper, we establish the following theorem. We first present singular vectors from distinct singular values of $n \times n$ matrix $A$ .

Theorem 1 Let $A = U^{H} Σ V \in C^{n \times n}$ with distinct singular values $σ_{i} (A)$ and $U = (u_{i j}), 1 \leq i, j \leq n$ be left singular vector matrix of $A$ and $V = (v_{l s}), 1 \leq l, s \leq n$ be right singular vector matrix of $A$ . Let ${¯ A}_{j}$ be $(n - 1) \times n$ matrix after deleting the j-th row of $A$ and $σ_{k} ({¯ A}_{j})$ be singular values of ${¯ A}_{j}$ . Let ${^A}_{s}$ be $n \times (n - 1)$ matrix after deleting the s-th column of $A$ and $σ_{t} ({^A}_{s})$ be singular values of ${^A}_{s}$ . Then

| u_{i j} |^{2} = \frac{n - 1 \prod k = 1 (σ_{i}^{2} (A) - σ_{k}^{2} ({¯ A}_{j}))}{n \prod k = 1, k \neq i (σ_{i}^{2} (A) - σ_{k}^{2} (A))}, 1 \leq i, j \leq n,

| v_{l s} |^{2} = \frac{n - 1 \prod t = 1 (σ_{l}^{2} (A) - σ_{t}^{2} ({^A}_{s}))}{n \prod t = 1, t \neq l (σ_{l}^{2} (A) - σ_{t}^{2} (A))}, 1 \leq l, s \leq n .

Then we deduce singular vectors from singular values of $m \times n$ matrix $A$ .

Theorem 2 Let $A = U^{H} Σ V \in C_{r}^{m \times n}$ with singular values $σ_{1} (A) \geq σ_{2} (A) \geq \dots \geq σ_{r} (A) > 0, σ_{r + 1} (A) = \dots = σ_{min {m, n}} (A) = \dots = σ_{max {m, n}} (A) = 0$ and $U = (u_{i j}), 1 \leq i, j \leq m$ be left singular vector matrix of $A$ and $V = (v_{l s}), 1 \leq l, s \leq n$ be right singular vector matrix of $A$ . Let ${¯ A}_{j}$ be $(m - 1) \times n$ matrix after deleting the j-th row of $A$ and $σ_{k} ({¯ A}_{j})$ be singular values of ${¯ A}_{j}$ . Let ${^A}_{s}$ be $m \times (n - 1)$ matrix after deleting the s-th column of $A$ and $σ_{t} ({^A}_{s})$ be singular values of ${^A}_{s}$ . Then

| u_{i j} |^{2} m \prod k = 1, k \neq i (σ_{i}^{2} (A) - σ_{k}^{2} (A)) = m - 1 \prod k = 1 (σ_{i}^{2} (A) - σ_{k}^{2} ({¯ A}_{j})), 1 \leq i, j \leq m,

| v_{l s} |^{2} n \prod t = 1, t \neq l (σ_{l}^{2} (A) - σ_{t}^{2} (A)) = n - 1 \prod t = 1 (σ_{l}^{2} (A) - σ_{t}^{2} ({^A}_{s})), 1 \leq l, s \leq n .

We note that the identities in Theorems 1 and 2 are not given in the previous results. The organization of this paper is given as follows. In Section 2, we present the proofs of the two main theorems. The concluding remarks are given in Section 3.

2 The Proof

In this section, we give the proofs of Theorems 1 and 2.

Proof of Theorem 1.

Let singular value decompositions of

A

A = U^{H} Σ V

, where

U \in U^{n}, V \in U^{n}

. (i) Let

A = ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} a_{2} ⋮ a_{n} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎠

with

a_{1}, \dots, a_{n} \in C^{1 \times n} .

Since

A A^{H} = U^{H} Σ Σ^{H} U,

by Lemma 1 we have

| u_{i j} |^{2} n \prod k = 1, k \neq i (λ_{i} (A A^{H}) - λ_{k} (A A^{H})) = n - 1 \prod k = 1 (λ_{i} (A A^{H}) - λ_{k} (S_{j})), 1 \leq i, j \leq n,

(1)

where $S_{j}$ is the $(n - 1) \times (n - 1)$ submatrix of $A A^{H}$ that results from deleting the $j$ th column and the $j$ th row, with eigenvalues $λ_{k} (S_{j})$ . It is easy to see that

S_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} a_{1}^{H} & \dots & a_{1} a_{j - 1}^{H} & a_{1} a_{j + 1}^{H} & \dots & a_{1} a_{n}^{H} a_{2} a_{1}^{H} & \dots & a_{2} a_{j - 1}^{H} & a_{2} a_{j + 1}^{H} & \dots & a_{2} a_{n}^{H} \dots & \dots & \dots & \dots & \dots & \dots a_{j - 1} a_{1}^{H} & \dots & a_{j - 1} a_{j - 1}^{H} & a_{j - 1} a_{j + 1}^{H} & \dots & a_{j - 1} a_{n}^{H} a_{j + 1} a_{1}^{H} & \dots & a_{j + 1} a_{j - 1}^{H} & a_{j + 1} a_{j + 1}^{H} & \dots & a_{j + 1} a_{n}^{H} \dots & \dots & \dots & \dots & \dots & \dots a_{n} a_{1}^{H} & \dots & a_{n} a_{j - 1}^{H} & a_{n} a_{j + 1}^{H} & \dots & a_{n} a_{n}^{H} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ .

Let ${¯ A}_{j}$ be $(n - 1) \times n$ matrix after deleting the j-th row of $A$ and $σ_{k} ({¯ A}_{j})$ be singular values of ${¯ A}_{j}$ , then

{¯ A}_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} ⋮ a_{j - 1} a_{j + 1} ⋮ a_{n} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠

and

σ_{k}^{2} ({¯ A}_{j}) = λ_{k}^{2} ({¯ A}_{j} {¯ A}_{j}^{H}) .

(2)

Since $A$ has distinct singular values, by (1) and (2) we have

| u_{i j} |^{2} = \frac{n - 1 \prod k = 1 (σ_{i}^{2} (A) - σ_{k}^{2} ({¯ A}_{j}))}{n \prod k = 1, k \neq i (σ_{i}^{2} (A) - σ_{k}^{2} (A))}, 1 \leq i, j \leq n .

(ii) Let $A = (b_{1}, \dots, b_{n})$ with $b_{1}, \dots, b_{n} \in C^{n} .$ Since

A^{H} A = V^{H} Σ^{H} Σ V,

by Lemma 1 we have

| v_{i j} |^{2} n \prod k = 1, k \neq i (λ_{i} (A^{H} A) - λ_{k} (A^{H} A)) = n - 1 \prod k = 1 (λ_{i} (A^{H} A) - λ_{k} (H_{j})), 1 \leq i, j \leq n,

(3)

where $H_{j}$ is the $(n - 1) \times (n - 1)$ submatrix of $A^{H} A$ that results from deleting the $j$ th column and the $j$ th row, with eigenvalues $λ_{k} (H_{j})$ . It is easy to see that

H_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} b_{1}^{H} b_{1} & \dots & b_{1}^{H} b_{j - 1} & b_{1}^{H} b_{j + 1} & \dots & b_{1}^{H} b_{n} b_{2}^{H} b_{1} & \dots & b_{2}^{H} b_{j - 1} & b_{2}^{H} b_{j + 1} & \dots & b_{2}^{H} b_{n} \dots & \dots & \dots & \dots & \dots & \dots b_{j - 1}^{H} b_{1} & \dots & b_{j - 1}^{H} b_{j - 1} & b_{j - 1}^{H} b_{j + 1} & \dots & b_{j - 1}^{H} b_{n} b_{j + 1}^{H} b_{1} & \dots & b_{j + 1}^{H} b_{j - 1} & b_{j + 1}^{H} b_{j + 1} & \dots & b_{j + 1}^{H} b_{n} \dots & \dots & \dots & \dots & \dots & \dots b_{n}^{H} b_{1} & \dots & b_{n}^{H} b_{j - 1} & b_{n}^{H} b_{j + 1} & \dots & b_{n}^{H} b_{n} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ .

Let ${^A}_{j}$ be $n \times (n - 1)$ matrix after deleting the j-th row of $A$ and $σ_{k} ({^A}_{j})$ be singular values of ${^A}_{j}$ , then

{^A}_{j} = (b_{1}, \dots, b_{j - 1}, b_{j + 1}, \dots, b_{n})

and

σ_{k}^{2} ({^A}_{j}) = λ_{k}^{2} ({^A}_{j}^{H} {^A}_{j}) .

(4)

Since $A$ has distinct singular values, by (3) and (4) we have

| v_{l s} |^{2} = \frac{n - 1 \prod t = 1 (σ_{l}^{2} (A) - σ_{t}^{2} ({^A}_{s}))}{n \prod t = 1, t \neq l (σ_{l}^{2} (A) - σ_{t}^{2} (A))}, 1 \leq l, s \leq n .

This completes the proof.

By using the similar trick, we can show the results of Theorem 2.

Proof of Theorem 2. Let singular value decompositions of $A$ be $A = U^{H} Σ V$ , where $U \in U^{m}, V \in U^{n}$ . (i) Let $A = ⎛ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} a_{2} ⋮ a_{m} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎠$ with $a_{1}, \dots, a_{m} \in C^{1 \times n} .$ Since

A A^{H} = U^{H} Σ Σ^{H} U,

by Lemma 1 we have

| u_{i j} |^{2} m \prod k = 1, k \neq i (λ_{i} (A A^{H}) - λ_{k} (A A^{H})) = m - 1 \prod k = 1 (λ_{i} (A A^{H}) - λ_{k} ({~ S}_{j})), 1 \leq i, j \leq m,

(5)

where ${~ S}_{j}$ is the $(m - 1) \times (m - 1)$ submatrix of $A A^{H}$ that results from deleting the $j$ th column and the $j$ th row, with eigenvalues $λ_{k} ({~ S}_{j})$ . It is easy to see that

{~ S}_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} a_{1}^{H} & \dots & a_{1} a_{j - 1}^{H} & a_{1} a_{j + 1}^{H} & \dots & a_{1} a_{m}^{H} a_{2} a_{1}^{H} & \dots & a_{2} a_{j - 1}^{H} & a_{2} a_{j + 1}^{H} & \dots & a_{2} a_{m}^{H} \dots & \dots & \dots & \dots & \dots & \dots a_{j - 1} a_{1}^{H} & \dots & a_{j - 1} a_{j - 1}^{H} & a_{j - 1} a_{j + 1}^{H} & \dots & a_{j - 1} a_{m}^{H} a_{j + 1} a_{1}^{H} & \dots & a_{j + 1} a_{j - 1}^{H} & a_{j + 1} a_{j + 1}^{H} & \dots & a_{j + 1} a_{m}^{H} \dots & \dots & \dots & \dots & \dots & \dots a_{m} a_{1}^{H} & \dots & a_{m} a_{j - 1}^{H} & a_{m} a_{j + 1}^{H} & \dots & a_{m} a_{m}^{H} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ .

Let ${~ A}_{j}$ be $(m - 1) \times n$ matrix after deleting the j-th row of $A$ and $σ_{k} ({~ A}_{j})$ be singular values of ${~ A}_{j}$ , then

{~ A}_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} a_{1} ⋮ a_{j - 1} a_{j + 1} ⋮ a_{m} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠

and

σ_{k}^{2} ({~ A}_{j}) = λ_{k}^{2} ({~ A}_{j} ~ A_{j}^{H}) .

(6)

By (5) and (6) we have

| u_{i j} |^{2} m \prod k = 1, k \neq i (σ_{i}^{2} (A) - σ_{k}^{2} (A)) = m - 1 \prod k = 1 (σ_{i}^{2} (A) - σ_{k}^{2} ({¯ A}_{j})), 1 \leq i, j \leq m .

(ii) Let $A = (b_{1}, \dots, b_{n})$ with $b_{1}, \dots, b_{n} \in C^{m} .$ Since

A^{H} A = V^{H} Σ^{H} Σ V,

by Lemma 1 we have

| v_{i j} |^{2} n \prod k = 1, k \neq i (λ_{i} (A^{H} A) - λ_{k} (A^{H} A)) = n - 1 \prod k = 1 (λ_{i} (A^{H} A) - λ_{k} (H_{j})), 1 \leq i, j \leq n,

(7)

where $H_{j}$ is the $(n - 1) \times (n - 1)$ submatrix of $A^{H} A$ that results from deleting the $j$ th column and the $j$ th row, with eigenvalues $λ_{k} (H_{j})$ . It is easy to see that

H_{j} = ⎛ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎜ ⎝ \begin{matrix} b_{1}^{H} b_{1} & \dots & b_{1}^{H} b_{j - 1} & b_{1}^{H} b_{j + 1} & \dots & b_{1}^{H} b_{n} b_{2}^{H} b_{1} & \dots & b_{2}^{H} b_{j - 1} & b_{2}^{H} b_{j + 1} & \dots & b_{2}^{H} b_{n} \dots & \dots & \dots & \dots & \dots & \dots b_{j - 1}^{H} b_{1} & \dots & b_{j - 1}^{H} b_{j - 1} & b_{j - 1}^{H} b_{j + 1} & \dots & b_{j - 1}^{H} b_{n} b_{j + 1}^{H} b_{1} & \dots & b_{j + 1}^{H} b_{j - 1} & b_{j + 1}^{H} b_{j + 1} & \dots & b_{j + 1}^{H} b_{n} \dots & \dots & \dots & \dots & \dots & \dots b_{n}^{H} b_{1} & \dots & b_{n}^{H} b_{j - 1} & b_{n}^{H} b_{j + 1} & \dots & b_{n}^{H} b_{n} \end{matrix} ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎠ .

Let ${^A}_{j}$ be $n \times (n - 1)$ matrix after deleting the j-th row of $A$ and $σ_{k} ({^A}_{j})$ be singular values of ${^A}_{j}$ , then

{^A}_{j} = (b_{1}, \dots, b_{j - 1}, b_{j + 1}, \dots, b_{n})

and

σ_{k}^{2} ({^A}_{j}) = λ_{k}^{2} ({^A}_{j}^{H} {^A}_{j}) .

(8)

By (7) and (8) we have

| v_{l s} |^{2} = \frac{n - 1 \prod t = 1 (σ_{l}^{2} (A) - σ_{t}^{2} ({^A}_{s}))}{n \prod t = 1, t \neq l (σ_{l}^{2} (A) - σ_{t}^{2} (A))}, 1 \leq l, s \leq n .

This completes the proof.

3 Concluding Remarks

In [1], Denton et al. revisited the identity of eigenvalue-eigenvector for Hermitian matrices. This identity has been discovered by many researchers in the literature. In this paper, we derived similar identity for singular vectors and singular values of general matrices. However, we find these results have not been studied previously. As a future research work, it may be useful to apply this new identity formula to matrix perturbation problems and the calculation of arbitrary singular values and singular vector instead of all singular values and singular vectors.

References

[1] P. Denton, S. Parke, T. Tao and X. Zhang, Eigenvectors from Eigenvalues: a Survey of a Basic Identity in Linear Algebra, arXiv:1908.03795.
[2] R. Thompson, Principal Submatrices IX: Interlacing Inequalities for Singular Values of Submatrices, Linear Algebra and its Applications, V5 (1972), pp. 1-12.

Singular Vectors From Singular Values

Authors

A note on perturbation analysis for T-product based tensor singular values

The Spectrum of the Singular Values of Z-Shaped Graph Matrices

On the ℓ^∞-norms of the Singular Vectors of Arbitrary Powers of a Difference Matrix with Applications to Sigma-Delta Quantization

On a question of Haemers regarding vectors in the nullspace of Seidel matrices

Symbolic determinant identity testing and non-commutative ranks of matrix Lie algebras

A rounding error analysis of the joint bidiagonalization process with applications to the GSVD computation

Computing singular elements modulo squares

1 Introduction

2 The Proof

3 Concluding Remarks

References

Singular Vectors From Singular Values

Authors

Related Research

A note on perturbation analysis for T-product based tensor singular values

The Spectrum of the Singular Values of Z-Shaped Graph Matrices

On the ℓ^∞-norms of the Singular Vectors of Arbitrary Powers of a Difference Matrix with Applications to Sigma-Delta Quantization

On a question of Haemers regarding vectors in the nullspace of Seidel matrices

Symbolic determinant identity testing and non-commutative ranks of matrix Lie algebras

A rounding error analysis of the joint bidiagonalization process with applications to the GSVD computation

Computing singular elements modulo squares

This week in AI

1 Introduction

2 The Proof

3 Concluding Remarks

References