Research Article | Open Access

Volume 2014 |Article ID 831321 | https://doi.org/10.1155/2014/831321

Hongcai Yin, Huamin Zhang, "Least Squares Based Iterative Algorithm for the Coupled Sylvester Matrix Equations", Mathematical Problems in Engineering, vol. 2014, Article ID 831321, 8 pages, 2014. https://doi.org/10.1155/2014/831321

# Least Squares Based Iterative Algorithm for the Coupled Sylvester Matrix Equations

Accepted26 May 2014
Published18 Jun 2014

#### Abstract

By analyzing the eigenvalues of the related matrices, the convergence analysis of the least squares based iteration is given for solving the coupled Sylvester equations and in this paper. The analysis shows that the optimal convergence factor of this iterative algorithm is 1. In addition, the proposed iterative algorithm can solve the generalized Sylvester equation . The analysis demonstrates that if the matrix equation has a unique solution then the least squares based iterative solution converges to the exact solution for any initial values. A numerical example illustrates the effectiveness of the proposed algorithm.

#### 1. Introduction

Matrix equations arise in systems and control, such as Lyapunov matrix equation and Riccati equation. How to solve these matrix equations becomes a major field of the matrix computations . Main points of this field are the decompositions and transformations of the matrices, eigenvalues and eigenvectors, and Krylov subspace . Other points contain the algorithms and the convergence analysis. The algorithms provided a set of operation steps, and according to these steps it can find a solution of a matrix equation within finite steps in given error bounds . The convergence analysis offered more details of an algorithm, and in general, these details indicated some new research areas .

The direct method and the indirect method are the two main approaches of solving the matrix equations . With the development of the requirement of the computation and the matrix theory, the indirect method or iterative method becomes the main approach of solving the matrix equations [19, 20].

Iterative method is active in studying other engineering problems. Particularly, the iterative method can be used to identify systems  and estimate parameter of systems . For example, iterative method is useful in the identification  and parameter estimation  of linear and nonlinear systems .

The Jacobi and Gauss-Seidel iterative methods were discussed in the literature. By extending the Jacobi and Gauss-Seidel iterative methods, Ding and his coworkers recently presented a large family of the least squares based iterative methods for solving the matrix equations and . It has been proved that these least squares based iterative solutions always converge fast to the exact ones as long as the unique solutions exist. But the range of the convergence factor is still open. Motivated by the importance of this algorithm, we develop a new way to prove the convergence of the least squares based iterative algorithm for the coupled Sylvester matrix equations and . According to the new proof, we obtain the optimal convergence factor of this iterative algorithm and extend the iterative algorithm to solve the generalized Sylvester matrix equation . The algorithm in this paper can be extended to other more general and complex matrix equations .

The paper is organized as follows. Section 2 gives some preliminaries. Section 3 presents a new proof to the least squares iterative algorithm for solving the coupled Sylvester matrix equations and . Section 4 revises this algorithm to solve the equation . Section 5 gives an example to illustrate the effectiveness of the proposed results. Finally, we offer some concluding remarks in Section 6.

#### 2. Basic Preliminaries

Some symbols and lemmas are introduced first. is the identity matrix of size . is an identity matrix with an appropriate size. denotes the eigenvalue set of matrix . denotes the matrix determinant. denotes the Frobenius norm of and is defined as . For an matrix denotes an -dimensional vector formed by represents the Kronecker product of matrices and . A formula involved the vec-operator and the Kronecker product is .

If matrix is invertible, then is symmetric and idempotent. To be more specific, we have the following lemma.

Lemma 1. For the symmetric matrix , there exists an orthogonal matrix such that Moreover, .

#### 3. The Coupled Sylvester Matrix Equations

In this section, we will prove the convergence of the least squares iterative algorithm for solving the following coupled Sylvester matrix equations: where , , and are known and are to be determined. By using the hierarchical identification principle, the least squares based iterative algorithm for solving (4) is given by  Here, and To initialize the algorithm, we take and as some small real matrices, for example, with being an matrix whose elements are all 1.

Theorem 2. If (4) has unique solutions and , then for any initial values and the iterative solutions and given by iteration in (5) converge to and ; that is, or the error matrices and converge to zero; in this case, the optimal convergence factor is .

Proof. Define two error matrices Using (4) and (5) gives Expanding (9) gives Simplifying (10) gives Using the formula gives Set Using these symbols, a compact form of (12) is Set The eigenvalues of inside the unit circle will complete the proof. Let be an eigenvalue of ; we will show that , or, if , then .
Consider the following characteristic polynomial of : It is not hard to show that by calculation. Since it follows from the formula that On the other hand, since (4) has a unique solution, it follows that the matrix is invertible. A determinant expansion shows that So it gives . Since expanding it gives Setting , it can be rewritten as According to Lemma 1, there exists an orthogonal matrix such that Using (25) can be manipulated to get If , then there exist the nonzero vectors that satisfy , which can be written as Since and are orthogonal matrices, it follows that where and . Then from (28), it gives Using (26) gives According to (29) and (30), (27) can be manipulated to get So it gives Suppose that , where , . From Schur decomposition theorem, there exists a decomposition , where is an orthogonal matrix, , and is a strictly upper triangular matrix. Then from (23), it gives It follows that . Since , we obtain , . Thus, .
Next, we determine the optimal convergence factor. From and one gets . Set . Taking absolute values of these eigenvalues, the optimal convergence factor satisfies Equation (35) is equivalent to . Solving it gives . The proof is completed.

#### 4. The Generalized Sylvester Matrix Equation AXB + CXD = F

In this section, we use iteration in (5) to solve the generalized Sylvester matrix equation. Consider the following equation: where , , and are given constant matrices and is the unknown matrix to be solved. The following conclusion is obvious.

Equation (36) has a unique solution if and only if In this case, the unique solution is given by .

Setting and , (36) can be equivalently expressed as If then (38) has a unique solution. It is easy to show that if or , then (38) is equivalent to (36). Next, we show that if or , then (39) is equivalent to (37). That is, we have the following determinant result: According to Theorem 2, (38) can be solved by iteration in (5), and from or , (36) can be solved.

#### 5. Example

In this section, an example is offered to illustrate the convergence of the proposed iterative algorithm.

Example 1. Consider the coupled Sylvester matrix equations in the form of (4) with The unique solution is found to be
Taking as the initial iterative values and using iteration (5) to compute and , the iterative values of and are shown in Table 1 with the relative error The effect of changing the convergence factor is illustrated in Figure 1.

 (%) 1 2.41386 5.12921 15.19476 −3.29213 8.34835 1.59243 4.18906 7.37550 58.29997 2 3.78243 4.89986 8.50024 3.36107 7.11113 1.69190 4.40964 7.01546 8.34339 3 3.76982 4.86436 9.81323 3.24974 8.02454 1.93498 4.94575 6.99503 6.82277 4 3.98646 4.99976 8.94082 3.91276 7.87581 1.98403 4.92193 7.02174 1.09451 5 3.96846 4.97726 9.13074 3.92885 8.00448 1.99063 4.99536 6.99758 0.92202 6 3.99906 5.00066 8.99011 3.98870 7.98094 2.00140 4.99013 7.00441 0.15912 7 3.99573 4.99642 9.02215 3.99419 8.00093 1.99865 4.99957 6.99955 0.14124 8 3.99996 5.00016 8.99824 3.99851 7.99694 2.00071 4.99874 7.00081 0.02497 9 3.99940 4.99943 9.00381 3.99973 8.00018 1.99980 4.99996 6.99992 0.02344 10 4.00000 5.00003 8.99969 3.99980 7.99949 2.00018 4.99983 7.00015 0.00413 11 3.99991 4.99991 9.00066 4.00004 8.00003 1.99997 5.00000 6.99999 0.00405 12 4.00000 5.00001 8.99994 3.99997 7.99991 2.00004 4.99998 7.00003 0.00070 Solution 4.00000 5.00000 9.00000 4.00000 8.00000 2.00000 5.00000 7.00000

From Table 1 and Figure 1, we find that the relative error goes to zero with the increasing of the iterative times. This shows that the proposed iterative algorithm is effective. In addition, Figure 1 shows that the optimal convergence factor . This indicates that the result of the optimal convergence suggested in this paper is correct.

#### 6. Conclusions

This paper proved the convergence of the least squares based iterative algorithm of the coupled Sylvester matrix equations and , and the proof determined the range of the convergence factor and the optimal convergence factor. The suggested algorithm can also be used to solve the generalized Sylvester equation . An example indicated that the iterative solution given by the least squares based iterative algorithm converges fast to its exact solution under proper conditions.

#### Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

#### Acknowledgment

This work was supported by the National Natural Science Foundation of China (no. 6110218).

1. X. P. Sheng, Y. F. Su, and G. L. Chen, “A modification of minimal residual iterative method to solve linear systems,” Mathematical Problems in Engineering, vol. 2009, Article ID 794589, 9 pages, 2009.
2. A. J. Liu and G. L. Chen, “On the hermitian positive definite solutions of nonlinear matrix equation ${X}^{s}+{A}^{*}{X}^{-{t}_{1}}A+{B}^{*}{X}^{-{t}_{2}}B=Q$,” Mathematical Problems in Engineeering, vol. 2011, Article ID 163585, 18 pages, 2011. View at: Publisher Site | Google Scholar
3. H. M. Zhang, “Iterative solutions of a set of matrix equations by using the hierarchical identification principle,” Abstract and Applied Analysis, vol. 2014, Article ID 649524, 10 pages, 2014. View at: Publisher Site | Google Scholar | MathSciNet
4. H. M. Zhang and F. Ding, “A property of the eigenvalues of the symmetric positive definite matrix and the iterative algorithm for coupled Sylvester matrix equations,” Journal of the Franklin Institute, vol. 351, no. 1, pp. 340–357, 2014. View at: Publisher Site | Google Scholar | MathSciNet
5. M. Hochbruck and G. Starke, “Preconditioned Krylov subspace methods for Lyapunov matrix equations,” SIAM Journal on Matrix Analysis and Applications, vol. 16, no. 1, pp. 156–171, 1995.
6. J. Yu, Q.-W. Wang, and C.-Z. Dong, “(Anti-)Hermitian generalized (Anti-)hamiltonian solution to a system of matrix equations,” Mathematical Problems in Engineering, vol. 2014, Article ID 539215, 13 pages, 2014. View at: Publisher Site | Google Scholar | MathSciNet
7. B. Zhou and G.-R. Duan, “On the generalized Sylvester mapping and matrix equations,” Systems & Control Letters, vol. 57, no. 3, pp. 200–208, 2008.
8. B. Zhou and G.-R. Duan, “An explicit solution to the matrix equation $AX-XF=BY$,” Linear Algebra and Its Applications, vol. 402, no. 1, pp. 345–366, 2005.
9. B. Zhou and G.-R. Duan, “A new solution to the generalized Sylvester matrix equation $AV-EVF=BW$,” Systems & Control Letters, vol. 55, no. 3, pp. 193–198, 2006.
10. B. Zhou and G.-R. Duan, “Parametric solutions to the generalized Sylvester matrix equation $AX-XF=BY$ and the regulator equation $AX-XF=BY+R$,” Asian Journal of Control, vol. 9, no. 4, pp. 475–483, 2007. View at: Publisher Site | Google Scholar | MathSciNet
11. B. Zhou and G.-R. Duan, “Solutions to generalized Sylvester matrix equation by Schur decomposition,” International Journal of Systems Science, vol. 38, no. 5, pp. 369–375, 2007.
12. Q. B. Liu and G. L. Chen, “Convergence analysis of preconditioned AOR iterative method for linear systems,” Mathematical Problems in Engineering, vol. 2010, Article ID 341982, 14 pages, 2010.
13. Y.-Q. Bai, T.-Z. Huang, and M.-M. Yu, “Convergence of a generalized USOR iterative method for augmented systems,” Mathematical Problems in Engineering, vol. 2013, Article ID 326169, 6 pages, 2013. View at: Publisher Site | Google Scholar | MathSciNet
14. M. Dehghan and M. Hajarian, “Two algorithms for finding the Hermitian reflexive and skew-Hermitian solutions of Sylvester matrix equations,” Applied Mathematics Letters, vol. 24, no. 4, pp. 444–449, 2011.
15. M. Dehghan and M. Hajarian, “Analysis of an iterative algorithm to solve the generalized coupled Sylvester matrix equations,” Applied Mathematical Modelling, vol. 35, no. 7, pp. 3285–3300, 2011.
16. M. Dehghan and M. Hajarian, “An iterative algorithm for solving a pair of matrix equations $AYB=E$, $CYD=F$ over generalized centro-symmetric matrices,” Computers & Mathematics with Applications, vol. 56, no. 12, pp. 3246–3260, 2008. View at: Publisher Site | Google Scholar | MathSciNet
17. M. Dehghan and M. Hajarian, “Finite iterative algorithms for the reflexive and anti-reflexive solutions of the matrix equation ${A}_{1}{X}_{1}{B}_{1}+{A}_{2}{X}_{2}{B}_{2}=C$,” Mathematical and Computer Modelling, vol. 49, no. 9-10, pp. 1937–1959, 2009.
18. M. Dehghan and M. Hajarian, “An efficient algorithm for solving general coupled matrix equations and its application,” Mathematical and Computer Modelling, vol. 51, no. 9-10, pp. 1118–1134, 2010.
19. F. Ding and T. Chen, “Gradient based iterative algorithms for solving a class of matrix equations,” IEEE Transactions on Automatic Control, vol. 50, no. 8, pp. 1216–1221, 2005. View at: Publisher Site | Google Scholar | MathSciNet
20. M. Dehghan and M. Hajarian, “An iterative method for solving the generalized coupled Sylvester matrix equations over generalized bisymmetric matrices,” Applied Mathematical Modelling, vol. 34, no. 3, pp. 639–654, 2010.
21. F. Ding, X. M. Liu, H. B. Chen, and G. Y. Yao, “Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems,” Signal Processing, vol. 97, pp. 31–39, 2014. View at: Publisher Site | Google Scholar
22. P. Shi, X. L. Luan, and F. Liu, “${H}_{\infty }$ filtering for discrete-time systems with stochastic incomplete measurement and mixed delays,” IEEE Transactions on Industrial Electronics, vol. 59, no. 6, pp. 2732–2739, 2012. View at: Publisher Site | Google Scholar
23. F. Ding, “Combined state and least squares parameter estimation algorithms for dynamic systems,” Applied Mathematical Modelling, vol. 38, no. 1, pp. 403–412, 2014. View at: Publisher Site | Google Scholar | MathSciNet
24. X. L. Luan, P. Shi, and F. Liu, “Stabilization of networked control systems with random delays,” IEEE Transactions on Industrial Electronics, vol. 58, no. 9, pp. 4323–4330, 2011. View at: Google Scholar
25. X. L. Luan, S. Y. Zhao, and F. Liu, “${H}_{\infty }$ control for discrete-time Markov jump systems with uncertain transition probabilities,” IEEE Transactions on Automatic Control, vol. 58, no. 6, pp. 1566–1572, 2013. View at: Publisher Site | Google Scholar | MathSciNet
26. Y. J. Liu, F. Ding, and Y. Shi, “An efficient hierarchical identification method for general dual-rate sampled-data systems,” Automatica, vol. 50, no. 3, pp. 962–970, 2014. View at: Publisher Site | Google Scholar | MathSciNet
27. L. Xie and H. Z. Yang, “Interactive parameter estimation for output error moving average systems,” Transactions of the Institute of Measurement and Control, vol. 35, no. 1, pp. 34–43, 2013. View at: Publisher Site | Google Scholar
28. F. Ding, “State filtering and parameter identification for state space systems with scarce measurements,” Signal Processing, vol. 104, pp. 369–380, 2014. View at: Publisher Site | Google Scholar
29. J. Ding, F. Ding, X. P. Liu, and G. Liu, “Hierarchical least squares identification for linear SISO systems with dual-rate sampled-data,” IEEE Transactions on Automatic Control, vol. 56, no. 11, pp. 2677–2683, 2011. View at: Publisher Site | Google Scholar | MathSciNet
30. L. Xie and H. Z. Yang, “Gradient-based iterative identification for nonuniform sampling output error systems,” Journal of Vibration and Control, vol. 17, no. 3, pp. 471–478, 2011.
31. D. Q. Wang, “Least squares-based recursive and iterative estimation for output error moving average systems using data filtering,” IET Control Theory & Applications, vol. 5, no. 14, pp. 1648–1657, 2011. View at: Publisher Site | Google Scholar | MathSciNet
32. X. G. Liu and J. Lu, “Least squares based iterative identification for a class of multirate systems,” Automatica, vol. 46, no. 3, pp. 549–554, 2010.
33. Y. J. Liu, J. Sheng, and R. F. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010.
34. J. Ding, C. X. Fan, and J. X. Lin, “Auxiliary model based parameter estimation for dual-rate output error systems with colored noise,” Applied Mathematical Modelling, vol. 37, no. 6, pp. 4051–4058, 2013. View at: Publisher Site | Google Scholar | MathSciNet
35. D. Q. Wang, F. Ding, and L. Ximei, “Least squares algorithm for an input nonlinear system with a dynamic subspace state space model,” Nonlinear Dynamics, vol. 75, no. 1-2, pp. 49–61, 2014.
36. J. H. Li, “Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration,” Applied Mathematics Letters, vol. 26, no. 1, pp. 91–96, 2013.
37. D. Q. Zhu and M. Kong, “Adaptive fault-tolerant control of non-linear systems: an improved CMAC-based fault learning approach,” International Journal of Control, vol. 80, no. 10, pp. 1576–1594, 2007.
38. D. Q. Zhu, J. Bai, and Q. Liu, “Multi-fault diagnosis method for sensor system based on PCA,” Sensors, vol. 10, no. 1, pp. 241–253, 2010. View at: Google Scholar
39. D. Q. Zhu, J. Liu, and S. X. Yang, “Particle swarm optimization approach to thruster fault-tolerant control of unmanned underwater vehicles,” International Journal of Robotics and Automation, vol. 26, no. 3, pp. 426–432, 2011. View at: Google Scholar
40. J. Ding, L. L. Han, and X. M. Chen, “Time series AR modeling with missing observations based on the polynomial transformation,” Mathematical and Computer Modelling, vol. 51, no. 5-6, pp. 527–536, 2010.
41. Y. J. Liu, Y. S. Xiao, and X. L. Zhao, “Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009.
42. Y. B. Hu, B. L. Liu, Q. Zhou, and C. Yang, “Recursive extended least squares parameter estimation for Wiener nonlinear systems with moving average noises,” Circuits, Systems, and Signal Processing, vol. 33, no. 2, pp. 655–664, 2014. View at: Publisher Site | Google Scholar | MathSciNet
43. Y. B. Hu, “Iterative and recursive least squares estimation algorithms for moving average systems,” Simulation Modelling Practice and Theory, vol. 34, pp. 12–19, 2013. View at: Publisher Site | Google Scholar
44. D. Q. Zhu, Q. Liu, and Z. Hu, “Fault-tolerant control algorithm of the manned submarine with multi-thruster based on quantum-behaved particle swarm optimisation,” International Journal of Control, vol. 84, no. 11, pp. 1817–1829, 2011.
45. D. Q. Zhu, Y. Zhao, and M. Z. Yan, “A bio-inspired neurodynamics based backstepping path-following control of an AUV with ocean current,” International Journal of Robotics and Automation, vol. 27, no. 3, pp. 280–287, 2012. View at: Google Scholar
46. D. Q. Zhu, H. Huang, and S. X. Yang, “Dynamic task assignment and path planning of multi-auv system based on an improved self-organizing map and velocity synthesis method in 3D underwater workspace,” IEEE Transactions on Cybernetics, vol. 43, no. 2, pp. 504–514, 2013. View at: Publisher Site | Google Scholar
47. F. Ding and T. Chen, “Iterative least-squares solutions of coupled Sylvester matrix equations,” Systems & Control Letters, vol. 54, no. 2, pp. 95–107, 2005.
48. F. Ding and T. Chen, “On iterative solutions of general coupled matrix equations,” SIAM Journal on Control and Optimization, vol. 44, no. 6, pp. 2269–2284, 2006.
49. F. Ding, P. X. Liu, and J. Ding, “Iterative solutions of the generalized Sylvester matrix equations by using the hierarchical identification principle,” Applied Mathematics and Computation, vol. 197, no. 1, pp. 41–50, 2008.
50. L. Xie, J. Ding, and F. Ding, “Gradient based iterative solutions for general linear matrix equations,” Computers & Mathematics with Applications, vol. 58, no. 7, pp. 1441–1448, 2009.
51. J. Ding, Y. J. Liu, and F. Ding, “Iterative solutions to matrix equations of the form ${A}_{i}X{B}_{i}={F}_{i}$,” Computers & Mathematics with Applications, vol. 59, no. 11, pp. 3500–3507, 2010.
52. L. Xie, Y. J. Liu, and H. Z. Yang, “Gradient based and least squares based iterative algorithms for matrix equations $AXB+{CX}^{T}D=F$,” Applied Mathematics and Computation, vol. 217, no. 5, pp. 2191–2199, 2010.
53. H. M. Zhang and F. Ding, “On the Kronecker products and their applications,” Journal of Applied Mathematics, vol. 2013, Article ID 296185, 8 pages, 2013.