Iterative Methods for Nonlinear Equations or Systems and Their Applications 2014
View this Special IssueResearch Article  Open Access
Approximating the Inverse of a Square Matrix with Application in Computation of the MoorePenrose Inverse
Abstract
This paper presents a computational iterative method to find approximate inverses for the inverse of matrices. Analysis of convergence reveals that the method reaches ninthorder convergence. The extension of the proposed iterative method for computing MoorePenrose inverse is furnished. Numerical results including the comparisons with different existing methods of the same type in the literature will also be presented to manifest the superiority of the new algorithm in finding approximate inverses.
1. Introduction
The main purpose of this paper is to present an efficient method in terms of speed of convergence, while its convergence can be easily achieved and also be economic for large sparse matrices possessing sparse inverses. We also discuss the extension of the new scheme for finding the MoorePenrose inverse of singular and rectangular matrices.
Finding a matrix inverse is important in some practical applications such as finding a rational approximation for the FermiDirac functions in the density functional theory [1], because it conveys significant features of the problems dealing with.
We further mention that in some certain circumstances, the computation of a matrix inverse is necessary. For example, there are many ways to encrypt a message, whereas the use of coding has become particularly significant in recent years. One way to encrypt or code a message is to use matrices and their inverses. Indeed, consider a fixed invertible matrix . Convert the message into a matrix such that is possible to perform. Send the message generated by . At the other end, they will need to know in order to decrypt or decode the message sent.
The direct methods such as Gaussian elimination with partial pivoting or LU decomposition require a reasonable time to compute the inverse when the size of the matrices is high. To illustrate further, the Gaussian elimination with partial pivoting method cannot be highly parallelized and this restricts its applicability in some cases. In contrary, Schulztype methods, which could be applied for large sparse matrices (possessing sparse inverses [2]) by preserving the sparsity feature and can be parallelized, are in focus in such cases.
Some known methods were proposed for the approximation of a matrix inverse, such as Schulz scheme. The oldest scheme, that is, the Schulz method (see [3]), is defined as wherein is the identity matrix. Note that [4] mentions that NewtonSchulz iterations can also be combined with wavelets or hierarchical matrices to compute the diagonal elements of independently.
The proposed iteration relies on matrix multiplications, which destroy sparsity, and therefore the Schulztype methods are less efficient for sparse inputs possessing dense inverses. However, a numerical dropping is usually applied to to keep the approximated inverse sparse. Such a strategy is useful for preconditioning.
To provide more iterations from this classification of methods, we remember that W. Li and Z. Li in [5] proposed
In 2011, Li et al. in [6] represented (2) in the following form: and also proposed another iterative method for finding as comes next:
Much of the application of these solvers (specially the Schulz method) for structured matrices was investigated in [7] while the authors in [8] showed that the matrix iteration of Schulz is numerically stable. Further discussion about such iterative schemes might be found at [9–12].
The Schulztype matrix iterations are dependent on the initial matrix too much. In general, we construct the initial guess for square matrices as follows. For a general matrix satisfying no structure, we choose as in [13] or , where is the identity matrix, and should adaptively be determined such that . For diagonally dominant (DD) matrices, we choose as in [14] , wherein is the diagonal entry of . And for a symmetric positive definite matrix and by using [15], we choose , whereas is the Frobenius norm.
For rectangular or singular matrices, one may choose or , based on [16]. Also, we could choose , where or with [17]. Note that these choices could also be considered for square matrices. However, the most efficient way for producing (for square nonsingular matrices) is the hybrid approach presented in Algorithm 1 of [18].
The rest of this paper is organized as follows. The main contributions of this article are given in Sections 2 and 3. In Section 2, we analyze a new scheme for matrix inversion while a discussion about the applicability of the scheme for MoorePenrose inverse will be given in Section 3. Section 4 will discuss the performance and the efficiency of the new proposed method numerically in comparison with the other schemes. Finally, concluding remarks are presented in Section 5.
2. Main Result
The derivation of different Schulztype methods for matrix inversion relies on iterative (one or multipoint) methods for the solution of nonlinear equations [19, 20]. For instance, imposing Newton’s iteration on the matrix equation would result in (1), as fully discussed in [21]. Here, we apply the following new nonlinear solver on the matrix equation : Note that (7) is novel and constructed in such a way to produce a new matrix iterative method for finding generalized inverses efficiently. Now, the following iteration method for matrix inversion could be obtained:
The obtained scheme includes matrix power, which is certainly of too much cost. To remedy this, we rewrite the obtained iteration as efficiently as possible to also reduce the number of matrixmatrix multiplications in what follows: whereas is the identity matrix and the sequence of matrix iterates converges to for a good initial guess.
It should be remarked that the convergence of any order for nonsingular square matrices is generated in Section 6 of Chapter 2 of [22], whereas the general way for the rectangular matrices was discussed in Chapter 5 of [23] and the recent paper of [24]. In those constructions, always a convergence order will be attained by times of matrixmatrix products, such as (1) which reaches the order 2 using two matrixmatrix multiplications.
Two important matters must be mentioned at this moment to ease up the perception of why a higher order (efficient) method such as (9) with 7 matrixmatrix products to reach at least the convergence order 9 is practical. First, by following the comparative index of informational efficiency for inverse finders [25], defined by , wherein and stand for the convergence order and the number of matrixmatrix products, then the informational index for (9), that is, , beats its other competitors, of (1), of (2)(3), and of (4). And second, the significance of the new scheme will be displayed in its implementation. To illustrate further, such iterations depend totally on the initial matrices. Though there are certain and efficient ways for finding , in general such initial approximations take a high number of iterations (see e.g., Figure 3, the blue color) to arrive at the convergence phase. On the other hand, each cycle of the implementation of such Schulztype methods includes one stopping criterion based on the use of a matrix norm, and this would impose further burden and load in general, for the low order methods in contrast to the high order methods, such as (9). Because the computation of a matrix norm (usually for dense complex matrices and for large sparse matrices) takes time and therefore higher number of steps/iterations (which is the result of lower order methods), it will be costlier than the lower number of steps/iterations of high order methods. Hence, the higher order (efficient) iterations are mostly better solvers in terms of computational time to achieve the desired accuracy.
After this complete discussion of the need and efficacy of the new scheme (9), we are about to prove the convergence behavior of (9) theoretically in what follows.
Theorem 1. Let be a nonsingular complex or real matrix. If the initial approximation satisfies then the iterative method (9) converges with at least ninth order to .
Proof. Let (10) hold, and for the sake of simplicity assume that and . It is straightforward to have Hence, by taking an arbitrary norm from both sides of (11), we obtain The rest of the proof is similar to Theorem of [26], and it is hence omitted. Finally, , when , and thus , as . So the sequence is strictly monotonic decreasing. Moreover, if we denote , as the error matrix in the iterative procedure (9), then we could easily obtain the following error inequality: The error inequality (13) reveals that the iteration (9) converges with at least ninth order to . The proof is now complete.
Some features of the new scheme (9) are as follows.(1)The new method can be taken into consideration for finding approximate inverse (preconditioners) of complex matrices. Using a dropping strategy, we could keep the sparsity of a matrix to be used as a preconditioner. Such an action will be used throughout this paper for sparse matrices using the command Chop[exp, tolerance] in Mathematica, to preserve the sparsity of the approximate inverses , for any .(2)Unlike the traditional direct solvers such as GEPP, the new scheme has the ability to be parallelized.(3)In order to further reduce the computational burden of the matrixbymatrix multiplications per computing step of the new algorithm when dealing with sparse matrices, in the new scheme by using Sparse Array command in Mathematica, we will preserve the sparsity features of the output approximate inverse in a reasonable computational time. Additionally, for some special matrices, such as Toeplitz or Vandermonde types of matrices, further computational savings are possible as discussed by Pan in [27]. See for more information [28].(4)If the initial guess commutes with the original matrix, then all matrices in the sequence satisfy the same property of commutation. Notice that only some initial matrices guarantee that in all cases the sequence commutes with the matrix .
In the next section, we present some analytical discussion about the fact that the new method (9) could also be used for computing the MoorePenrose generalized inverse.
3. An Iterative Method for MoorePenrose Inverse
It is well known in the literature that the MoorePenrose inverse of a complex matrix (also called pseudoinverse), denoted by , is a matrix satisfying the following conditions: wherein is the conjugate transpose of . This inverse uniquely exists.
Various numerical solution methods have been developed for approximating MoorePenrose inverse (see e.g., [29]). The most comprehensive review of such iterations could be found in [17], while the authors mentioned that iterative Schulztype methods can be taken into account for finding pseudoinverse. For example, it is known that (1) converges to the pseudoinverse in the general case if , where and denotes the spectral radius.
Accordingly, it is known that the new scheme (9) converges to the MoorePenrose inverse. In order validate this fact analytically, we provide some important theoretics in what follows.
Lemma 2. For the sequence generated by the iterative Schulztype method (9) and the initial matrix (6), for any , it holds that
Proof. The proof of this lemma is based on mathematical induction. Such a procedure is similar to Lemma 3.1 of [30], and it is hence omitted.
Theorem 3. For the rectangular complex matrix and the sequence generated by (9), for any , using the initial approximation (6), the sequence converges to the pseudoinverse with ninth order of convergence.
Proof. We consider , as the error matrix for finding the MoorePenrose inverse. Then, the proof is similar to the proof of Theorem 4 in [18] and it is hence omitted. We finally have Thus, ; that is, the obtained sequence of (9) converges to the MoorePenrose inverse as . This ends the proof.
Theorem 4. Considering the same assumptions as in Theorem 3, then the iterative method (9) has asymptotical stability for finding the MoorePenrose generalized inverse.
Proof. The steps of proving the asymptotical stability of (9) are similar to those which have recently been taken for a general family of methods in [31]. Hence, the proof is omitted.
Remark 5. It should be remarked that the generalization of our proposed scheme for generalized outer inverses, that is, , is straight forward according to the recent work [32].
Remark 6. The new iteration (9) is free from matrix power in its implementation and this allows one to apply it for finding generalized inverses easily.
4. Computational Experiments
Using the computer algebra system Mathematica 8 [33], we now compare our iterative algorithm with other Schulztype methods.
For numerical comparisons in this section, we have used the methods (1), (3), (4), and (9). We implement the above algorithms on the following examples using the builtin precision in Mathematica 8.
Example 7. Let us consider the large complex sparse matrix of the size with 79512 nonzero elements possessing a sparse inverse as in Algorithm 1 (), where the matrix plot of showing its structure has been drawn in Figure 1(a).

(a)
(b)
Methods like (9) are powerful in finding an approximate inverse or in finding robust approximate inverse preconditioners in low number of steps, in which the output form of the approximate inverse is also sparse.
In this test problem, the stopping criterion is . Table 1 shows the number of iterations for different methods. As the programs were running, we found the running time using the command Absolute Timing to report the elapsed CPU time (in seconds) for this experiment. Note that the initial guess has been constructed using in Example 7. The plot of the approximate inverse obtained by applying (9) has been portrayed in Figure 1(b). The reason which made (4) the worse method is in the fact of computing matrix power which is time consuming for sparse matrices. In the tables of this paper “No. of nonzero ele.” stands for the number of nonzero elements.
In Algorithm 2, we provide the written Mathematica 8 code of the new scheme (9) in this example to clearly reveals the simplicity of the new scheme in finding approximate inverses using a threshold .

Example 8. Let us consider a large sparse matrix (with 18601 nonzero elements) as shown in Algorithm 3.

Table 2 shows the number of iterations for different methods in order to reveal the efficiency of the proposed iteration with a special attention to the elapsed time. The matrix plot of in this case showing its structure alongside its approximate inverse has been drawn in Figure 2, respectively. Note that in Example 8, we have used an initial value constructed by . The stopping criterion is same as the previous example.
(a)
(b)
The new method (9) is totally better in terms of the number of iterations and the computational time. In this paper, the computer specifications are Microsoft Windows XP Intel(R), Pentium(R) 4, CPU 3.20 GHz, and 4 GB of RAM.
Matrices used up to now are large and sparse enough to clarify the applicability of the new scheme. Anyhow, for some classes of problems, there are collections of matrices that are used when proving new iterative linear system solvers or new preconditioners, such as Matrix Market, http://math.nist.gov/MatrixMarket/. In the following test we compare the computational time of solving a practical problem using the above source.
Example 9. Consider the linear system of , in which is defined by A = Example Data [“Matrix”, “Bai/pde900”] and the right hand side vector is b = Table [1., {i, 1, 900}]. In Table 3, we compare the consuming time (in seconds) of solving this system by GMRES solver and its left preconditioned system . In Table 3, for example, PGMRES(9) shows that the linear system , in which has been calculated by (9), is solved by GMRES solver. The tolerance is the residual norm to be less than in this test. The results support fully the efficiency of the new scheme (9) in constructing robust preconditioners.
Example 10. This experiment evaluates the applicability of the new method for finding MoorePenrose inverse of 20 random sparse matrices (possessing sparse pseudoinverses) as shown in Algorithm 4.

Note that the identity matrix should then be an matrix and the approximate pseudoinverses would be of the size . In this example, the initial approximate MoorePenrose inverse is constructed by for each random test matrix. And the stopping criterion is . We only compared methods (1) denoted by “Schulz”, (3) denoted by “KMS,” and the new iterative scheme (9) denoted by “PM.”
The results for the number of iterations and the running time are compared in Figures 34. They show a clear advantage of the new scheme in finding the MoorePenrose inverse.
5. Conclusions
In this paper, we have developed an iterative method in inverse finding of matrices, which has the capability to be applied on real or complex matrices with preserving the sparsity feature in matrixbymatrix multiplications using a threshold. This allows us to interestingly reduce the computational time and usage memory in applying the new iterative method. We have shown that the suggested method (9) reaches ninth order of convergence. The extension of the scheme for pseudoinverse has also been discussed in the paper. The applicability of the new scheme was illustrated numerically in Section 4.
Finally, according to the numerical results obtained and the concrete application of such solvers, we can conclude that the new method is efficient.
The new scheme (9) has been tested for matrix inversion. However, we believe that such iterations could also be applied for finding the inverse of an integer in modulo [34]. Following this idea will, for sure, open a new topic of research which is a combination of Numerical Analysis and Number Theory. We will focus on this trend of research for future studies.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
The authors thank the referees for their valuable comments and for the suggestions to improve the readability of the paper.
References
 R. B. Sidje and Y. Saad, “Rational approximation to the FermiDirac function with applications in density functional theory,” Numerical Algorithms, vol. 56, no. 3, pp. 455–479, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Soleymani and P. S. Stanimirović, “A higher order iterative method for computing the Drazin inverse,” The Scientific World Journal, vol. 2013, Article ID 708647, 11 pages, 2013. View at: Publisher Site  Google Scholar
 G. Schulz, “Iterative Berechnung der Reziproken matrix,” Zeitschrift für angewandte Mathematik und Mechanik, vol. 13, pp. 57–59, 1933. View at: Google Scholar
 L. Lin, J. Lu, R. Car, and E. Weinan, “Multipole representation of the Fermi operator with application to the electronic structure analysis of metallic systems,” Physical Review B—Condensed Matter and Materials Physics, vol. 79, no. 11, Article ID 115133, 2009. View at: Publisher Site  Google Scholar
 W. Li and Z. Li, “A family of iterative methods for computing the approximate inverse of a square matrix and inner inverse of a nonsquare matrix,” Applied Mathematics and Computation, vol. 215, no. 9, pp. 3433–3442, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H.B. Li, T.Z. Huang, Y. Zhang, X.P. Liu, and T.X. Gu, “Chebyshevtype methods and preconditioning techniques,” Applied Mathematics and Computation, vol. 218, no. 2, pp. 260–270, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 W. Hackbusch, B. N. Khoromskij, and E. E. Tyrtyshnikov, “Approximate iterations for structured matrices,” Numerische Mathematik, vol. 109, no. 3, pp. 365–383, 2008. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 T. Söderström and G. W. Stewart, “On the numerical properties of an iterative method for computing the MoorePenrose generalized inverse,” SIAM Journal on Numerical Analysis, vol. 11, pp. 61–74, 1974. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 X. Liu and Y. Qin, “Successive matrix squaring algorithm for computing the generalized inverse ${A}_{T,S}^{(2)}$,” Journal of Applied Mathematics, vol. 2012, Article ID 262034, 12 pages, 2012. View at: Publisher Site  Google Scholar
 X. Liu and S. Huang, “Proper splitting for the generalized inverse ${A}_{T,S}^{(2)}$ and its application on Banach spaces,” Abstract and Applied Analysis, vol. 2012, Article ID 736929, 9 pages, 2012. View at: Google Scholar  Zentralblatt MATH  MathSciNet
 X. Liu and F. Huang, “Higherorder convergent iterative method for computing the generalized inverse over Banach spaces,” Abstract and Applied Analysis, vol. 2013, Article ID 356105, 5 pages, 2013. View at: Google Scholar  MathSciNet
 X. Liu, H. Jin, and Y. Yu, “Higherorder convergent iterative method for computing the generalized inverse and its application to Toeplitz matrices,” Linear Algebra and Its Applications, vol. 439, no. 6, pp. 1635–1650, 2013. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 J. Rajagopalan, An iterative algortihm for inversion of matrices [M.S. thesis], Concordia University, Quebec, Canada, 1996.
 L. Grosz, “Preconditioning by incomplete block elimination,” Numerical Linear Algebra with Applications, vol. 7, no. 78, pp. 527–541, 2000. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 G. Codevico, V. Y. Pan, and M. Van Barel, “Newtonlike iteration based on a cubic polynomial for structured matrices,” Numerical Algorithms, vol. 36, no. 4, pp. 365–380, 2004. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 V. Pan and R. Schreiber, “An improved Newton iteration for the generalized inverse of a matrix, with applications,” SIAM Journal on Scientific and Statistical Computing, vol. 12, no. 5, pp. 1109–1130, 1991. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 A. BenIsrael and T. N. E. Greville, Generalized Inverses, Springer, 2nd edition, 2003. View at: MathSciNet
 F. Khaksar Haghani and F. Soleymani, “A new highorder stable numerical method for matrix inversion,” The Scientific World Journal, vol. 2014, Article ID 830564, 10 pages, 2014. View at: Publisher Site  Google Scholar
 F. Soleymani, S. Karimi Vanani, H. I. Siyyam, and I. A. AlSubaihi, “Numerical solution of nonlinear equations by an optimal eighthorder class of iterative methods,” Annali dell'Universitá di Ferrara. Sezione VII. Scienze Matematiche, vol. 59, no. 1, pp. 159–171, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 X. Wang and T. Zhang, “Highorder Newtontype iterative methods with memory for solving nonlinear equations,” Mathematical Communications, vol. 19, pp. 91–109, 2014. View at: Google Scholar
 H. Hotelling, “Analysis of a complex of statistical variables into principal components,” Journal of Educational Psychology, vol. 24, no. 6, pp. 417–441, 1933. View at: Publisher Site  Google Scholar
 E. Isaacson and H. B. Keller, Analysis of Numerical Methods, John Wiley & Sons, New York, NY, USA, 1966. View at: MathSciNet
 E. V. Krishnamurthy and S. K. Sen, Numerical Algorithms—Computations in Science and Engineering, Affiliated EastWest Press, New Delhi, India, 2007. View at: MathSciNet
 L. Weiguo, L. Juan, and Q. Tiantian, “A family of iterative methods for computing MoorePenrose inverse of a matrix,” Linear Algebra and Its Applications, vol. 438, no. 1, pp. 47–56, 2013. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 F. Soleymani, “A fast convergent iterative solver for approximate inverse of matrices,” Numerical Linear Algebra with Applications, 2013. View at: Publisher Site  Google Scholar
 M. Zaka Ullah, F. Soleymani, and A. S. AlFhaid, “An efficient matrix iteration for computing weighted MoorePenrose inverse,” Applied Mathematics and Computation, vol. 226, pp. 441–454, 2014. View at: Publisher Site  Google Scholar  MathSciNet
 V. Y. Pan, Structured Matrices and Polynomials: Unified Superfast Algorithms, Springer, Birkhäauser, Boston, Mass, USA, 2001. View at: Publisher Site  MathSciNet
 M. Miladinović, S. Miljković, and P. Stanimirović, “Modified SMS method for computing outer inverses of Toeplitz matrices,” Applied Mathematics and Computation, vol. 218, no. 7, pp. 3131–3143, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 H. Chen and Y. Wang, “A family of higherorder convergent iterative methods for computing the MoorePenrose inverse,” Applied Mathematics and Computation, vol. 218, no. 8, pp. 4012–4016, 2011. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
 A. R. Soheili, F. Soleymani, and M. D. Petković, “On the computation of weighted MoorePenrose inverse using a highorder matrix method,” Computers & Mathematics with Applications, vol. 66, no. 11, pp. 2344–2351, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 F. Soleymani and P. S. Stanimirović, “A note on the stability of a $p$th order iteration for finding generalized inverses,” Applied Mathematics Letters, vol. 28, pp. 77–81, 2014. View at: Publisher Site  Google Scholar  MathSciNet
 P. S. Stanimirović and F. Soleymani, “A class of numerical algorithms for computing outer inverses,” Journal of Computational and Applied Mathematics, vol. 263, pp. 236–245, 2014. View at: Publisher Site  Google Scholar  MathSciNet
 S. Wolfram, The Mathematica Book, Wolfram Media, 5th edition, 2003. View at: MathSciNet
 M. P. Knapp and C. Xenophontos, “Numerical analysis meets number theory: using rootfinding methods to calculate inverses mod ${p}^{n}$,” Applicable Analysis and Discrete Mathematics, vol. 4, no. 1, pp. 23–31, 2010. View at: Publisher Site  Google Scholar  Zentralblatt MATH  MathSciNet
Copyright
Copyright © 2014 F. Soleymani et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.