Research Article | Open Access
A Modified Gradient Based Algorithm for Solving Matrix Equations
In this paper, we develop a modified gradient based algorithm for solving matrix equations . Different from the gradient based method introduced by Xie et al., 2010, the information generated in the first half-iterative step is fully exploited and used to construct the approximate solution. Theoretical analysis shows that the new method converges under certain assumptions. Numerical results are given to verify the efficiency of the new method.
Consider a linear matrix equation of the following form: where , , , , and are the given constant matrices and is the unknown matrix to be solved. A Sylvester equation is the special case of (1) with and , the notation is the identity matrix of . Such kind of problems frequently arise from many areas of applications in control and system theory , stability of linear systems , analysis of bilinear systems , power systems , signal and image processing , and so forth.
The exact solutions of matrix equations, such as Lyapunov and Sylvester matrix equations, can be obtained from matrix inversion by using the Kronecker product. The drawback of this approach is that considerable computational costs and storage requirements are needed, so that this approach is only applicable for small sized Sylvester equations. Some direct methods have also been proposed in [6–9] which are based on the idea of transforming the coefficient matrix into a Schur or Hessenberg form, by which the original equation can be solved by a backward substitution.
In the numerical linear community, iterative methods are becoming more and more popular. Several iterative schemes for Sylvester equations have been proposed; see, for example, [10–15]. Recently, some efficient gradient based and least squares based iterative algorithms for solving generalized Sylvester equations and coupled (general coupled) Sylvester equations have been presented [16–28]. The basic idea of these approaches is based on a hierarchical identification principle [16–18], which regards the unknown matrix as the system parameter matrix to be identified and then constructs a recursive formula to approximate the unknown solution. Particularly, for general linear matrix equations of form (1), it is illustrated in [3, 4] that the unknown matrix to be identified can be computed by a gradient based iterative algorithm. The convergence properties of the methods are also investigated in . In this paper, a modified gradient based iterative algorithm is proposed for solving linear matrix equations of form (1). The information generated in the first half-iterative step is fully exploited and used to construct the approximate solution by the modified method. The convergence condition of the method is analyzed. The numerical performance of the method is compared with the algorithms in [3, 4]. Numerical results show that the new method is efficient and robust.
The paper is organized as follows. In Section 2, the gradient based iterative method is recalled, and the modified gradient based method is introduced and analyzed in Section 3. In Section 4, performance of the modified gradient based method is compared with the original one. Finally, we conclude the paper in Section 5.
2. A Brief Review of the Gradient Based Iterative Method
We firstly recall an iterative method proposed by Xie et al.  for solving (1). The basic idea is regarding (1) as two linear matrix equations as follows: Then, define two recursive sequences as follows: where is the iterative step size. The above procedures can be regarded as two separate iterative procedures for solving two matrix equations in (3).
With and at hand, then the th approximate solution can be defined by taking the average of two approximate solutions, that is, By selecting an appropriate initial approximate solution , and using to substitute in (4) and in (5), then the above (4)–(6) constitute the gradient based iterative method proposed in . It is shown in  that the gradient based iterative algorithm converges as long as where is the largest eigenvalue of .
According to lots of numerical experiments, GBI algorithm is computationally efficient. However, we observe that the GBI algorithm has some limitations. The convergent rate is slow and the stagnation will happen for ill-conditioned problem. Also, in , the authors pointed out that how to choose a best convergence factor is a subject to be studied and deserves further research. In this paper, we present the optimal convergence factor explicitly, and then propose a modified algorithm for solving the linear matrix equations (1).
3. A Modified Gradient Based Iterative Algorithm
The above GBI process can be accomplished by the following algorithm.
Algorithm 1 (see ). The gradient based iterative algorithm (GBI algorithm).(1)Give two initial approximate solutions and (2)for , until converges(3)(4)(5)(6)end.
In the step of computing , the last approximate solution has been computed. Hence, we can use the information of to update the and present a modification of GBI algorithm.
Algorithm 2. The modified gradient based iterative algorithm (MGBI algorithm).(1)Give two initial approximate solutions and (2)for , until converges(3)(4)(5)(6)(7)end.
Let , , , , , and
Lemma 3 (see ). The Sylvester equation given by (1) has a unique solution if and only if , in this case, the unique solution is given by The corresponding homogeneous equation has a unique zero solution .
The following result discusses the convergence conditions of the Algorithm 2.
Theorem 4. If the linear matrix equation (1) has a unique solution and then the iterative sequence generated by Algorithm 2 converges to ; that is, ; or the error converges to zero for any initial value .
Proof. In the following discussions, we always assume that the Frobenius matrix norm is used. For the clarity of proof, we introduce another variable instead of in the fifth iteration. Define the following error matrices: By using (1) and (10), the proof of the following equalities is trivial: Taking the Frobenius norm of both sides of (11), it follows that From , we have Obviously, . In fact, the iterative sequence , generated by Algorithm 2 can also be viewed as the sequence generated by the double-side iteration in , so . As , , we have It follows that or According to Lemma 3, we have as .
4. Numerical Experiments
Example 1. Consider the matrix equation with From (8), the exact solution is
The coefficient matrices used in this example are taken from . Taking , we apply the GBI algorithm and MGBI algorithm to compute ; the convergence factor is set to be in GBI algorithm and to be , in MGBI algorithm. The relative error is recorded and plotted in Figure 1 by MATLAB command semilogy. From the figure, we can see that the MGBI algorithm converges faster than the GBI algorithm.
Remark. The choice of the convergence factor is an important issue. We experimentally study its influence on the convergence. The effect of changing of the convergence factor for MGBI algorithms in Example 1 is illustrated in Figure 2. We see that is a better convergence factor. However, the convergence factor is problem dependent, so seeking a best convergence factor is still a difficult task.
Example 2. Suppose that , where Then, the solution of from (8) is The coefficient matrices used in this example are taken from . Taking , we apply the GBI algorithm and MGBI algorithm to compute , the convergence factor is set to be in GBI algorithm and to be in MGBI algorithm. The relative error is recorded in Figure 3. From the figure, we can also see that the MGBI algorithm converges faster than the GBI algorithm.
In this paper, a modified gradient based iteration (MGBI) method is proposed for linear matrix equation. The convergence of MGBI is analyzed. The choice of parameter is an important issue, and its influence is experimentally studied. The principle idea of this paper can be extended to the more general setting like generalized (coupled) Sylvester matrix equations.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
- R. Bhatia and P. Rosenthal, “How and why to solve the operator equation ,” The Bulletin of the London Mathematical Society, vol. 29, no. 1, pp. 1–21, 1997.
- B. N. Datta, Numerical Methods for Linear Control Systems, Elsevier Academic Press, 2003.
- L. Xie, Y. Liu, and H. Yang, “Gradient based and least squares based iterative algorithms for matrix equations AXB + CXTD = F,” Applied Mathematics and Computation, vol. 217, no. 5, pp. 2191–2199, 2010.
- L. Xie, J. Ding, and F. Ding, “Gradient based iterative solutions for general linear matrix equations,” Computers & Mathematics with Applications, vol. 58, no. 7, pp. 1441–1448, 2009.
- J. Ding, Y. Liu, and F. Ding, “Iterative solutions to matrix equations of the form ,” Computers & Mathematics with Applications, vol. 59, no. 11, pp. 3500–3507, 2010.
- G. H. Golub, S. Nash, and C. Van Loan, “A Hessenberg-Schur method for the problem ,” IEEE Transactions on Automatic Control, vol. 24, no. 6, pp. 909–913, 1979.
- R. H. Bartels and G. W. Stewart, “Algorithm 432: solution of the matrix equation AX - XB = C,” Communications of the ACM, vol. 15, pp. 820–826, 1972.
- D. C. Sorensen and Y. Zhou, “Direct methods for matrix Sylvester and Lyapunov equations,” Journal of Applied Mathematics, no. 6, pp. 277–303, 2003.
- W. H. Enright, “Improving the efficiency of matrix operations in the numerical solution of stiff ordinary differential equations,” Transactions on Mathematical Software, Association for Computing Machinery, vol. 4, no. 2, pp. 127–136, 1978.
- D. Calvetti and L. Reichel, “Application of ADI iterative methods to the restoration of noisy images,” SIAM Journal on Matrix Analysis and Applications, vol. 17, no. 1, pp. 165–186, 1996.
- D. Y. Hu and L. Reichel, “Krylov-subspace methods for the Sylvester equation,” Linear Algebra and its Applications, vol. 172, pp. 283–313, 1992.
- P. Kirrinnis, “Fast algorithms for the Sylvester equation ,” Theoretical Computer Science, vol. 259, no. 1-2, pp. 623–638, 2001.
- G. Starke and W. Niethammer, “SOR for ,” Linear Algebra and its Applications, vol. 154/156, pp. 355–375, 1991.
- Q. Niu, X. Wang, and L.-Z. Lu, “A relaxed gradient based algorithm for solving Sylvester equations,” Asian Journal of Control, vol. 13, no. 3, pp. 461–464, 2011.
- Z.-Z. Bai, “On Hermitian and skew-Hermitian splitting iteration methods for continuous Sylvester equations,” Journal of Computational Mathematics, vol. 29, no. 2, pp. 185–198, 2011.
- F. Ding and T. Chen, “On iterative solutions of general coupled matrix equations,” SIAM Journal on Control and Optimization, vol. 44, no. 6, pp. 2269-–2284, 2006.
- F. Ding and T. Chen, “Iterative least-squares solutions of coupled Sylvester matrix equations,” Systems & Control Letters, vol. 54, no. 2, pp. 95–107, 2005.
- F. Ding and T. Chen, “Gradient based iterative algorithms for solving a class of matrix equations,” Institute of Electrical and Electronics Engineers. Transactions on Automatic Control, vol. 50, no. 8, pp. 1216–1221, 2005.
- F. Ding, P. X. Liu, and J. Ding, “Iterative solutions of the generalized Sylvester matrix equations by using the hierarchical identification principle,” Applied Mathematics and Computation, vol. 197, no. 1, pp. 41–50, 2008.
- F. Ding, Y. Shi, and T. Chen, “Gradient-based identification methods for Hammerstein nonlinear ARMAX models,” Nonlinear Dynamics, vol. 45, no. 1-2, pp. 31–43, 2006.
- F. Ding, P. X. Liu, and G. Liu, “Gradient based and least-squares based iterative identification methods for OE and OEMA systems,” Digital Signal Processing, vol. 20, no. 3, pp. 664–677, 2010.
- Y. Liu, J. Sheng, and R. Ding, “Convergence of stochastic gradient estimation algorithm for multivariable ARX-like systems,” Computers & Mathematics with Applications, vol. 59, no. 8, pp. 2615–2627, 2010.
- Y. Liu, Y. Xiao, and X. Zhao, “Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model,” Applied Mathematics and Computation, vol. 215, no. 4, pp. 1477–1483, 2009.
- F. Ding, G. Liu, and X. P. Liu, “Parameter estimation with scarce measurements,” Automatica, vol. 47, no. 8, pp. 1646–1655, 2011.
- F. Ding and T. Chen, “Performance analysis of multi-innovation gradient type identification methods,” Automatica, vol. 43, no. 1, pp. 1–14, 2007.
- F. Ding, Y. J. Liu, and B. Bao, “Gradient based and least squares based iterative estimationalgorithms for multi-input multi-output systems,” Proceedings of the Institution of MechanicalEngineers I: Journal of Systems and Control Engineering, vol. 226, no. 1, pp. 43–55, 2012.
- X. Wang, L. Dai, and D. Liao, “A modified gradient based algorithm for solving Sylvester equations,” Applied Mathematics and Computation, vol. 218, no. 9, pp. 5620–5628, 2012.
- J. Zhou, R. Wang, and Q. Niu, “A preconditioned iteration method for solving Sylvester equations,” Journal of Applied Mathematics, vol. 2012, Article ID 401059, 12 pages, 2012.
Copyright © 2014 Kanmin Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.