Research Article | Open Access
A Modification of Minimal Residual Iterative Method to Solve Linear Systems
We give a modification of minimal residual iteration (MR), which is 1V-DSMR to solve the linear system . By analyzing, we find the modifiable iteration to be a projection technique; moreover, the modification of which gives a better (at least the same) reduction of the residual error than MR. In the end, a numerical example is given to demonstrate the reduction of the residual error between the 1V-DSMR and MR.
One of the important computational problems in the applied science and engineering is the solution of nonsingular linear systems of equations:where is a symmetric positive definite matrix (referred to as an SPD matrix), is given, and is unknown. To solve this problem, usually an iterative method is spurred by demands, which can be found in excellent papers [1, 2]. Most of the existing practical iterative techniques for solving larger linear systems of (1.1) utilize a projection process in one way or another; see, for example, [3–9].
Projection techniques are presented in different forms in many other areas of scientific computing, which can be formulated in abstract Hilbert functional spaces and finite element spaces. Furthermore, projection techniques are the process in which one attempts to solve a set of equations by solving each separate equation by correcting so that it is small in some norm. The idea of projection process is to extract an approximate solution to (1.1) from a subspace of . Denote and the search subspace and the constraints subspace, respectively. Let be their dimension and be an initial guess to the solution of (1.1). A projection method onto the subspace and orthogonal to is a process which finds an approximation solution to (1.1) by imposing the Petrov-Galerkin conditions that belongs to the affine space and the new residual vector is orthogonal to , that is,
From this point of view, the basic iterative methods for solving (1.1), such as Gauss-Seidel Iteration (GS), Steepest Descent Iteration (SD), Minimal Residual Iteration (MR), and Residual Norm Steepest Descent Iteration (RNSD), all can be viewed as a special case of the projection techniques.
In , Ujević obtained a new iterative method for solving (1.1), which is considered as a modification of Gauss-Seidel method. In , Jing and Huang pointed that this iterative method is also a projection process and named this method as “one-dimensional double successive projection method” (referred to as 1D-DSPM). In the same paper , the authors obtained another iterative method, which is named as “two-dimensional double successive projection method” (referred to as 2D-DSPM). The theory indicates that 2D-DSPM gives a better reduction of error than 1D-DSPM.
2. Notations and Preliminaries
In this paper, we will consider the following linear system of equations:where is not a symmetric but a positive definite matrix of order is a given element, and unknown. For linear systems (2.1), we can use the classical minimal residual iteration to solve, which can be found in . Here, we will give a modification of minimal residual iteration to solve linear system (2.1). We call the modification method as one vector double successive MR (abbreviated 1V-DSMR) and compare reduction of the residual error at step between the modification iteration and original MR. Hence, we find that the modification iteration gives a better reduction of the residual error than the original MR. denotes a vector inner product between the vector .
We define the inner products as
In this subsection, we will recall minimal residual iterative method and give some properties of this iteration.
Algorithm 2.1. (1) Choose an initial guess solution to (2.1), .
(3) If , then stop; else,
(5) Go to step (3).
The minimal residual iteration can be interpreted with projection techniques. Here we represent the principles of this method in our uniform notation as follows:where .
If we choose and , then (1.2) turns to findwhere .
If we choose a special , then (2.6) is the minimal residual iteration (MR); up to now, it is clear that MR is a special case of projection methods.
For the MR, we have the following property.
3. An Interpretation of 1V-DSMR with Projection Technique
In this section, we will give the modification of the minimal residual iterative method, which is abbreviated to 1V-DSMR; we can present this method in our uniform notation as follows:where and .
We will have a two-step investigation of 1V-DSMR.
The first step is to choose and , then it turns into the proceeding of MR, so we have .
The next step is a similar way to choose and ; denote and (1.2) turns to findwhere .
If we choose a special , then (3.1) is a modification of the minimal residual iteration, which is named as 1V-DSMR; up to now, it is clear that 1V-DSMR is also a special case of projection methods.
As 1V-DSMR, we have the following relation of residual errors.
Next we will depict the comparison results with respect to residual error reduction between 1V-DSMR and MR.
Theorem 3.2. 1V-DSMR gives a better (at least the same) reduction of the residual error than MR.
Theorem 3.2 implies that the residual error of MR is bigger than that of 1V-DSMR at step if the residual vectors and at the th iteration are equal to each other.
4. A Particular Method of 1V-DSMR
In this section, particular and will be chosen, and an algorithm to interpret 1V-DSMR is obtained.
Since 1V-DSMR is a modification of minimal residual iteration, take . In general, may be chosen in different ways. Here, we choose a particular , then from (3.1), each step of 1V-DSMR is as follows:where andIn this case, the first step of 1V-DSMR is the same as minimal residual iteration.
The above results provide the following algorithm of 1V-DSMR.
Algorithm 4.1 (A particular implementation of 1V-DSMR in a generalized way). (1) Choose an initial guess solution of (2.1), .
(4) If , then stop; else,
(6) Goto step (4).
About Algorithm 4.1, we have the following basic property.
Theorem 4.2. If is a positive matrix, then the sequence of the iterations of Algorithm 4.1 converges to the solution of the linear system
Proof. From the equality (3.9), we get
This means that the sequence is a decreasing and bounded one. Thus, the sequence in question is convergent implying that the left-hand side tends to zero. Obviously, tends to zero, and the proof is complete.
5. Numerical Examples
In this section, we use examples to further examine the effectiveness and show the advantages of 1V-DSMR over MR.
We compare the numerical behavior of Algorithm 4.1 with Algorithm 2.1. All the tests are performed by MATLAB 7.0. Because of the influence of the error of roundoff, we regard the matrix as zero matrix if .
For convenience of comparison, consider the two-dimensional partial differential equations on the unit square region of the formwhere and are all given real valued function of and , which are as follows: , and .
Here, we use a five-point finite difference scheme to discretize the above problem with a uninform grid of mesh spacing in and directions, respectively; we can obtain a matrix of order as varies, which is called PDE matrix. Now, we take , then we get a matrix, which is called PDE900, and denoted by . It is easy to check that is real unsymmetrical and nonsingular.
If we take the coefficient matrix of linear system (2.1) as , the right vector , and initial iterative vector , then we use Algorithms 2.1 and 4.1 to compute the linear system (2.1), respectively. The comparison results between MR and 1V-DSMR are shown in Figure 1.
From Figure 1, we can see that the convergence velocity of Algorithm 4.1 is always faster than that of Algorithm 2.1. In fact, when we use Algorithm 2.1 to compute the linear system (2.1), we only need to iterate 814 steps, and the residual norm is . While using Algorithm 4.1, we only need to iterate 647 steps, and the residual norm is .
This project was granted financial support from Shanghai Science and Technology Committee (no. 062112065), Shanghai Priority Academic Discipline Foundation, The University Young Teacher Sciences Foundation of Anhui Province (no. 2006jql220zd) and PhD Program Scholarship Fund of ECNU2007.
- Y. Saad and H. A. van der Vorst, “Iterative solution of linear systems in the 20th century,” Journal of Computational and Applied Mathematics, vol. 123, no. 1-2, pp. 1–33, 2000.
- N. Ujević, “A new iterative method for solving linear systems,” Applied Mathematics and Computation, vol. 179, no. 2, pp. 725–730, 2006.
- Å. Björck and T. Elfving, “Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations,” BIT, vol. 19, no. 2, pp. 145–163, 1979.
- R. Bramley and A. Sameh, “Row projection methods for large nonsymmetric linear systems,” SIAM Journal on Scientific and Statistical Computing, vol. 13, no. 1, pp. 168–193, 1992.
- C. Brezinski, Projection Methods for Systems of Equations, vol. 7 of Studies in Computational Mathematics, North-Holland, Amsterdam, The Netherlands, 1997.
- C. Kamath and A. Sameh, “A projection method for solving nonsymmetric linear systems on multiprocessors,” Parallel Computing, vol. 9, no. 3, pp. 291–312, 1989.
- L. Lopez and V. Simoncini, “Analysis of projection methods for rational function approximation to the matrix exponential,” SIAM Journal on Numerical Analysis, vol. 44, no. 2, pp. 613–635, 2006.
- V. Simoncini, “Variable accuracy of matrix-vector products in projection methods for eigencomputation,” SIAM Journal on Numerical Analysis, vol. 43, no. 3, pp. 1155–1174, 2005.
- K. Tanabe, “Projection method for solving a singular system of linear equations and its applications,” Numerische Mathematik, vol. 17, pp. 203–214, 1971.
- Y.-F. Jing and T.-Z. Huang, “On a new iterative method for solving linear systems and comparison results,” Journal of Computational and Applied Mathematics, vol. 220, no. 1-2, pp. 74–84, 2008.
- Y. Saad, Iterative Methods for Sparse Linear Systems, SIAM, Philadelphia, Pa, USA, 2nd edition, 2003.
Copyright © 2009 Xingping Sheng et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.