Procrustes Problems for General, Triangular, and Symmetric Toeplitz Matrices

Abstract

The Toeplitz Procrustes problems are the least squares problems for the matrix equation 𝐴𝑋=𝐵 over some Toeplitz matrix sets. In this paper the necessary and sufficient conditions are obtained about the existence and uniqueness for the solutions of the Toeplitz Procrustes problems when the unknown matrices are constrained to the general, the triangular, and the symmetric Toeplitz matrices, respectively. The algorithms are designed and the numerical examples show that these algorithms are feasible.

1. Introduction

Consider the constrained least-squares optimization problems: min_𝑋∈𝑃‖𝐴𝑋−𝐵‖_𝐹, (1) where 𝐴,𝐵∈𝑅^𝑛×𝑛, 𝑃⊆𝑅^𝑛×𝑛, and ‖⋅‖_𝐹 denotes the Frobenius norm ‖𝐴‖_𝐹=tr𝐴^𝑇𝐴=^𝑚𝑛_𝑖=1𝑗=1𝑎²_𝑖𝑗^1/2. (2)

These problems have been investigated by a series of literatures. For instance, when 𝑃=𝑅^𝑛×𝑛, then 𝑋=𝐴⁺𝐵 is a solution (refer to [1]), while the symmetric matrix solution case and orthogonal matrix solution case are analyzed in [2] and [3] respectively. And the symmetric positive (semi-) definite least squares problem is discussed by Higham [4].

In this paper, the following Toeplitz least-squares problem are analyzed: min‖𝐴𝑋−𝐵‖²_𝐹,s.t.𝑋∈𝒯, (3) where 𝐴,𝐵∈𝑅^𝑛×𝑛 and 𝒯⊂𝑅^𝑛×𝑛 are the set of Toeplitz matrices, and a matrix 𝑇 is called a Toeplitz matrix if its entries satisfy 𝑇(𝑖,𝑗)=𝑇(𝑖+𝑘,𝑗+𝑘), for all 𝑖, 𝑗, and 𝑘 with 0⩽𝑖+𝑘⩽𝑛−1, 0⩽𝑗+𝑘⩽𝑛−1, 𝛼𝑇=⁰𝛼₁𝛼₂⋯𝛼_𝑛−1𝛼₋₁𝛼₀𝛼₁⋮⋮⋱⋱⋱𝛼₂⋱⋱𝛼₁𝛼_{−(𝑛−1)}⋯𝛼₋₁𝛼₀. (4)

In this work, we discuss problem (3) in details. In Section 2, the general Toeplitz problem is discussed, and in Section 3, the triangular Toeplitz problem and the symmetric Toeplitz problem are discussed, while in Section 4, we give algorithms and numerical examples.

We give some definition and lemmas first.

Definition 1 (see [5]). Matrix 𝐺_𝑛×𝑚 is called {1} inverse matrix of 𝐴_𝑚×𝑛 when 𝐴𝐺𝐴=𝐴, and it can be denoted by 𝐴⁻ or 𝐴{1}.

Lemma 2 (see [5]). Let 𝐴∈𝐶_𝑟^𝑚×𝑛; if there exist nonsingular matrices 𝑃∈𝐶^𝑛×𝑛 and 𝑅∈𝐶^𝑚×𝑚 such that 𝐼𝑅𝐴𝑃=^𝑟𝟎𝟎𝟎, (5) then the sufficient and necessary condition of 𝐺∈𝐴{1} is 𝐼𝐺=𝑃^𝑟𝑈𝑉W𝑅, (6) where 𝑈∈𝐶^{𝑟×(𝑚−𝑟)},𝑉∈𝐶^{(𝑛−𝑟)×𝑟}, and 𝑊∈𝐶^{(𝑛−𝑟)×(𝑚−𝑟)}.

Remark 3. When 𝑈=𝟎, 𝑉=𝟎, and 𝑊=𝟎, 𝐺 is denoted by 𝐴⁻_∗.

Lemma 4 (see [6]). A finite dimensional linear subspace of a normed linear space contains at least one point of minimum distance from a fixed point.

Lemma 5 (see [7]). Let 𝑓∶𝑅^𝑛→𝑅 be a convex function continuously differentiable, and then any local minimum point is also a global minimum point. And 𝑥^∗ is a solution of problem min𝑓(𝑥) if and only if 𝑥^∗ satisfies ∇𝑓(𝑥^∗)=0, where ∇ means the gradient of function 𝑓.

Lemma 6 (see [7]). The quadratic function 𝑓(𝑥)=(1/2)𝑋^𝑇𝑄𝑋+𝑞^𝑇𝑋+𝐶 is convex if and only if 𝑄 is positive semidefinite.

2. The General Toeplitz Problem

We transform problem (3) into the following equivalent form: 1𝐹≡2‖𝐴𝑋−𝐵‖²_𝐹=min,𝑋=^(𝑛−1)_{()𝑝=−𝑛−1}𝛼_𝑝𝐺_𝑝, (7) where 𝛼_𝑝∈𝑅, 𝐺_𝑝∈𝑅^𝑛×𝑛 are defined as 𝐺_𝑝𝑖𝑗=()()1if𝑗=𝑖+𝑝,0otherwise,𝑝=−𝑛−1,…,𝑛−1. (8) The objective function 𝐹 can be rewritten as 1𝐹=2tr(𝐴𝑋−𝐵)^𝑇(=1𝐴𝑋−𝐵)2tr𝑋^𝑇𝐴^𝑇𝐴𝑋+−tr𝐵^𝑇+1𝐴𝑋2tr𝐵^𝑇𝐵=𝐹₁+𝐹₂. (9) The first term of the right side in (9) can be expressed as 𝐹₁=12tr𝑋^𝑇𝐴^𝑇=1𝐴𝑋2tr^(𝑛−1)_{()𝑙=−𝑛−1}𝛼_𝑙𝐺_𝑙^𝑇𝐴^𝑇𝐴^(𝑛−1)_{()𝑘=−𝑛−1}𝛼_𝑘𝐺_𝑘=12^(𝑛−1)_{()𝑙,𝑘=−𝑛−1}tr𝐺^𝑇_𝑙𝐴^𝑇𝐴𝐺_𝑘𝛼_𝑙𝛼_𝑘=12^(𝑛−1)_{()𝑙,𝑘=−𝑛−1}tr𝐺^𝑇_𝑙𝐴^𝑇𝐴𝐺_𝑘+tr𝐺^𝑇_𝑘𝐴^𝑇𝐴𝐺_𝑙2𝛼_𝑙𝛼_𝑘=12𝑦^𝑇𝐶𝑦, (10) where 𝑦^𝑇=𝛼_{−(𝑛−1)},𝛼_{−(𝑛−2)},…,𝛼₀,…,𝛼_(𝑛−1),𝐶=𝐶_{𝑙𝑘(2𝑛−1)×(2𝑛−1)},𝐶_𝑙𝑘=12tr𝐺^𝑇_𝑙𝐴^𝑇𝐴𝐺_𝑘+tr𝐺^𝑇_𝑘𝐴^𝑇𝐴𝐺_𝑙,𝑙,𝑘=−(𝑛−1),…,(𝑛−1). (11) Also the second term of the right side in (9) can be expressed as 𝐹₂=−tr𝐵^𝑇+1𝐴𝑋2tr𝐵^𝑇𝐵=−^(𝑛−1)_{()𝑙=−𝑛−1}tr𝐵^𝑇𝐴𝐺_𝑙𝛼_𝑙+12tr𝐵^𝑇𝐵=−𝑏^𝑇1𝑦+2tr𝐵^𝑇𝐵, (12) where 𝑏^𝑇=(tr(𝐵^𝑇𝐴𝐺_{−(𝑛−1)}), tr(𝐵^𝑇𝐴𝐺_{−(𝑛−2)}),…, tr(𝐵^𝑇𝐴𝐺_(𝑛−1)).

It thus yields that 1𝐹=2𝑦^𝑇𝐶𝑦−𝑏^𝑇1𝑦+2tr𝐵^𝑇𝐵. (13)

Taking partial derivative on 𝐹 about 𝛼_𝑙 gives 𝜕𝐹𝜕𝛼_𝑙=𝜕𝐹₁𝜕𝛼_𝑙+𝜕𝐹₂𝜕𝛼_𝑙=𝐶_{𝑙,−(𝑛−1)},𝐶_{𝑙,−(𝑛−2)},…,𝐶_{𝑙,(𝑛−1)}𝑦−tr𝐵^𝑇𝐴𝐺_(𝑙)=𝐶^𝑇_𝑙𝑦−𝑏_𝑙,𝑙=−(𝑛−1),…,(𝑛−1). (14)

By the first order necessary condition, 𝜕𝐹𝜕𝛼_{−(𝑛−1)}⋮=0,𝜕𝐹𝜕𝛼_(𝑛−1)=0, (15) we obtain 𝐶𝑦=𝑏. (16) Therefore the following theorem holds.

Theorem 7. The solution of problem (3) exists, and its general form can be expressed as 𝑋=^(𝑛−1)_{()𝑙=−𝑛−1}𝛼_𝑙𝐺_𝑙, (17) where (𝛼_{−(𝑛−1)},𝛼_{−(𝑛−2)},…,𝛼₀,…,𝛼_(𝑛−1))^𝑇=𝑦=𝐶⁻𝑏+(𝐼−𝐶⁻𝐶)𝑧, for all 𝑧∈𝑅^𝑛,𝐶⁻ means {1} inverse of 𝐶.

Proof. Let 𝒯^′={𝑇∈𝑅^𝑛×𝑛∣𝑇=𝐴∑^(𝑛−1)_{𝑝=−(𝑛−1)}𝛼_𝑝𝐺_𝑝}, where 𝛼_𝑝∈𝑅, and then the subspace 𝒯^′ is a normed linear space under Frobenius norm. It follows from Lemma 4 that there exists at least one matrix 𝑇 in 𝒯^′ such that ‖𝐴𝑋−𝐵‖²_𝐹=min. So, by the definition of 𝑇, there exists at least one scalar 𝛼^𝑇=(𝛼_{−(𝑛−1)},𝛼_{−(𝑛−2)},…,𝛼₀,…,𝛼_(𝑛−1)) constituting matrix 𝑇, which means we can rewrite 𝑇 as 𝑇=𝐴𝑋, where 𝑋∈𝒯. Therefore the solution of problem (3) exists.
It is apparent from (13) that the objective function 𝐹 is a quadratic function. Besides, by the following discussion, 𝐹 is also a convex function. Since 𝐴^𝑇𝐴 is positive semi-definite, it follows that, for any 𝑋∈𝑅^𝑛×𝑛, the expression of 𝐹₁ in (10) holds 𝐹₁=12𝑦^𝑇1𝐶𝑦=2tr𝑋^𝑇𝐴^𝑇𝐴𝑋⩾0. (18) In other words, 𝐶 is a positive semidefinite matrix. Then according to Lemma 6, 𝐹 is a convex function. Since 𝐹 is continuously differentiable, we then get that any solution of (16) is also a solution of (7) by Lemma 5. Clearly, the solution of linear system of (16) is 𝑦=𝐶⁻𝑏+(𝐼−𝐶⁻𝐶)𝑧,∀𝑧∈𝑅^2𝑛−1 (19) (see [5]). The required solution 𝑋 of problem (7) is then given by 𝑋=∑^(𝑛−1)_{𝑙=−(𝑛−1)}𝛼_𝑙𝐺_𝑙, where 𝑦=(𝛼_{−(𝑛−1)},𝛼_{−(𝑛−2)},…,𝛼₀,…,𝛼_(𝑛−1))^𝑇. The theorem follows by the equivalence of problem (3) and problem (7).

Theorem 8. Problem (3) has a unique solution if and only if 𝐶 has full rank. In this case, the unique solution is 𝑋=^(𝑛−1)_{()𝑙=−𝑛−1}𝛼_𝑙𝐺_𝑙, (20) where (𝛼_{−(𝑛−1)},𝛼_{−(𝑛−2)},…,𝛼₀,…,𝛼_(𝑛−1))^𝑇=𝑦=𝐶⁻¹𝑏.

Proof (necessity). If 𝐶 has full rank, then the linear system of (16) has a unique solution 𝑦=𝐶⁻¹𝑏; therefore the solution 𝑋 of problem (7) is determined uniquely, so is problem (3).
Sufficiency follows from the conversely procedure of necessity.

Corollary 9. If A has full rank in column, then the solution of problem (3) is unique.

3. The Triangular Toeplitz Problem and the Symmetric Toeplitz Problem

In this section, we discuss the triangular Toeplitz problem and symmetric Toeplitz problem.

3.1. The Triangular Toeplitz Problem

By the upper triangular Toeplitz problem, as the definition of [6], we mean the minimization problem min‖𝐴𝑋−𝐵‖²_𝐹s.t.𝑋∈𝒯_𝑢, (21) where 𝐴,𝐵∈𝑅^𝑚×𝑛, 𝒯_𝑢 are the subspace of upper triangular Toeplitz matrices, with elements of the general form 𝛼𝑋=¹𝛼₂𝛼₃⋯𝛼_𝑛0𝛼₁𝛼₂⋮⋮⋱⋱⋱𝛼₃⋱⋱𝛼₂0⋯0𝛼₁. (22)

We transform problem (21) into the following problem 1𝐹≡2‖𝐴𝑋−𝐵‖²_𝐹=min,𝑋=^𝑛_𝑝=1𝛼_𝑝𝐻_𝑝, (23) where 𝐻_𝑝𝑖𝑗=1if𝑗=𝑖+𝑝−1,0otherwise,𝑝=1,…,𝑛. (24) The proof procedure is similar to that in problem (7). In this case, the related function 𝐹, the unknown 𝑦, the linear system of equations, and the matrix 𝐶 as well as the scalar 𝑏 can be expressed as 1𝐹=2𝑦^𝑇𝐶_𝑢𝑦−𝑏^𝑇_𝑢1𝑦+2tr𝐵^𝑇𝐵,𝐶_𝑢𝑦=𝑏_𝑢,𝑦^𝑇=𝛼₁,𝛼₂,…,𝛼_𝑛,𝐶_𝑢=𝐶_{𝑙𝑘𝑛×𝑛},𝐶_𝑙𝑘=12tr𝐻^𝑇_𝑙𝐴^𝑇𝐴𝐻_𝑘+tr𝐻^𝑇_𝑘𝐴^𝑇𝐴𝐻_𝑙,𝑏𝑙,𝑘=1,2,…,𝑛,_𝑢=tr𝐵^𝑇𝐴𝐻₁,tr𝐵^𝑇𝐴𝐻₂,…,tr𝐵^𝑇𝐴𝐻_𝑛. (25)

Therefore we obtain a conclusion on upper triangular Toeplitz problem as follows.

Theorem 10. The solution of problem (21) exists, and its general form can be expressed as 𝑋=^𝑛_𝑙=1𝛼_𝑙𝐻_𝑙, (26) where (𝛼₁,𝛼₂,…,𝛼_𝑛)^𝑇=𝑦_𝑢=𝐶⁻_𝑢𝑏_𝑢+(𝐼−𝐶⁻_𝑢𝐶_𝑢)𝑧, for all 𝑧∈𝑅^𝑛, 𝐶⁻_𝑢 means {1} inverse of 𝐶_𝑢.

Theorem 11. Problem (21) has a unique solution if and only if 𝐶_𝑢 has full rank. In this case, the unique solution is 𝑋=^𝑛_𝑙=1𝛼_𝑙𝐻_𝑙, (27) where (𝛼₁,𝛼₂,…,𝛼_𝑛)^𝑇=𝐶_𝑢⁻¹𝑏_𝑢.

Corollary 12. If A has full rank in column, then the solution of problem (21) is unique.

Similarly, we can solve the lower triangular Toeplitz problem.

3.2. The Symmetric Toeplitz Problem

The symmetric Toeplitz problem is the following minimization problem: min‖𝐴𝑋−𝐵‖²_𝐹s.t.𝑋∈𝒯_𝑠, (28) where 𝐴,𝐵∈𝑅^𝑚×𝑛, 𝒯_𝑠 are the subspace of symmetric Toeplitz matrices, with elements of the general form: 𝛼𝑋=¹𝛼₂𝛼₃⋯𝛼_𝑛𝛼₂𝛼₁𝛼₂⋮𝛼₃𝛼₂⋱⋱𝛼₃⋮⋱⋱𝛼₂𝛼_𝑛⋯𝛼₃𝛼₂𝛼₁. (29) We consider the following objective function: 1𝐹≡2‖𝐴𝑋−𝐵‖²_𝐹=min,𝑋=^𝑛_𝑝=1𝛼_𝑝𝑄_𝑝, (30) where 𝑄_𝑝𝑖𝑗=1if𝑗−𝑖=𝑝−1,0otherwise,𝑝=1,…,𝑛. (31)

In this case, the related function 𝐹, the unknown 𝑦, the linear system of equations, and the matrix 𝐶 as well as the scalar 𝑏 can be expressed as 1𝐹=2𝑦^𝑇𝐶_𝑠𝑦−𝑏^𝑇_𝑠1𝑦+2tr𝐵^𝑇𝐵,𝐶_𝑠𝑦=𝑏_𝑠,𝑦^𝑇=𝛼₁,𝛼₂,…,𝛼_𝑛,𝐶_𝑠=𝐶_{𝑙𝑘𝑛×𝑛},𝐶_𝑙𝑘=12tr𝑄^𝑇_𝑙𝐴^𝑇𝐴𝑄_𝑘+tr𝑄^𝑇_𝑘𝐴^𝑇𝐴𝑄_𝑙,𝑏𝑙,𝑘=1,2,…,𝑛,_𝑠=tr𝐵^𝑇𝐴𝑄₁,tr𝐵^𝑇𝐴𝑄₂,…,tr𝐵^𝑇𝐴𝑄_𝑛. (32)

Therefore we obtain a conclusion about the symmetric Toeplitz problem.

Theorem 13. The solution of problem (28) exists, and its general form can be expressed as 𝑋=^𝑛_𝑙=1𝛼_𝑙𝑄_𝑙, (33) where (𝛼₁,𝛼₂,…,𝛼_𝑛)^𝑇=𝑦=𝐶⁻_𝑠𝑏_𝑠+(𝐼−𝐶⁻_𝑠𝐶_𝑠)𝑧, for all 𝑧∈𝑅^𝑛, and 𝐶⁻_𝑠 means {1} inverse of 𝐶_𝑠.

Theorem 14. Problem (28) has a unique solution if and only if 𝐶_𝑠 has full rank. In this case, the unique solution is 𝑋=^𝑛_𝑙=1𝛼_𝑙𝑄_𝑙, (34) where (𝛼₁,𝛼₂,…,𝛼_𝑛)^𝑇=𝑦=𝐶_𝑠⁻¹𝑏_𝑠.

Corollary 15. If A has full rank in column, then the solution of problem (28) is unique.

4. Computation and Examples

The derivation of general Toeplitz problem leads to the following computational algorithm.

Algorithm 16. (1)Generate matrices 𝐺_𝑖(𝑖=−(𝑛−1),…,(𝑛−1)) according to the definition in (7).(2)Compute the matrix 𝐶 and scalar 𝑏 by formula (11) and (12).(3)If 𝐶 is of full rank, then solving the linear system of (16) directly, to obtain the solution 𝑦=𝐶⁻¹𝑏; else calculate 𝐶⁻, forming 𝑦=𝐶⁻𝑏+(𝐼−𝐶⁻𝐶)𝑧, for all 𝑧∈𝑅^𝑛. In general, let 𝑈, 𝑉 and 𝑊 in Lemma 2 equal to 𝟎, we can obtain one of {1} inverse of 𝐶, that is 𝐶⁻_∗.(4)Compute 𝑋 from 𝑋=∑^(𝑛−1)_{𝑙=−(𝑛−1)}𝛼_𝑙𝐺_𝑙, where (𝛼_{−(𝑛−1)},…,𝛼_(𝑛−1))=𝑦^𝑇.

The algorithms for other special Toeplitz problems can be designed similarly.

We mention that, in the beginning of this paper, solutions for that two instances are actually obtained based on Algorithm 16. We can give another example of Toeplitz problem with matrix 𝐴 of full rank in column as follows. According to Corollaries 9, 12, and 15, the solution is unique.

Example 17. Consider the following ,.𝐴=−220−1100222132−2−1−1−1−121𝐵=0−2−321−2−3100−3−32022320−3 (35)
Using the algorithms mentioned above, computing the Toeplitz problems by Matlab, we obtain the following results.
The general Toeplitz matrix solution is 𝑋_∗=−0.40791−0.837960.080264−0.041414−0.55794−0.40791−0.837960.0802640.62335−0.55794−0.40791−0.837960.306560.62335−0.55794−0.40791. (36) The upper triangular Toeplitz matrix solution is 𝑋_𝑢=−0.37531−0.76827−0.069704−0.0798900.0−0.37531−0.76827−0.0697040.00.0−0.37531−0.768270.00.00.0−0.37531, (37) and the symmetric Toeplitz matrix solution is 𝑋_𝑠=−0.46064−0.644770.356590.27712−0.64477−0.46064−0.644770.356590.35659−0.64477−0.46064−0.644770.277120.35659−0.64477−0.46064. (38)

5. Conclusions

We have discussed the Toeplitz Procrustes problem including the general, triangular, and symmetric cases. After transforming the origin problem into a quadric form, we gain the general or unique solution with facility, by solving the linear system of equations.

Acknowledgments

The authors want to thank M. G. Eberle and M. C. Macle who have first proposed and studied the Toeplitz Procrustes problem. They also thank any referee for helpful suggestions or strengthening the theorems.