The Preconditioned SOR Iterative Method for Positive Definite Matrices

Abstract

We present several iterations for preconditioners introduced by Tarazaga and Cuellar (2009), and study the convergence of the method for solving a linear system whose coefficient matrix is positive definite matrices, and we also find that they complete very well with the SOR iteration, which is shown through numerical examples.

1. Introduction

For solving the large sparse linear system 𝐴𝑥=𝑏,𝑥,𝑏∈𝑅^𝑛, (1) where 𝐴∈𝑅^𝑛×𝑛 is a square nonsingular positive definite matrix, an iteration method is often considered. For any splitting, 𝐴=𝑀−𝑁 with det(𝑀)≠0, the basic iterative method for system (1) is 𝑥^𝑘+1=𝑀⁻¹𝑁𝑥^𝑘+𝑀⁻¹𝑏,𝑘=0,1,…. (2)

To improve the convergence rate of the basic iterative method, transform the original systems (1) into the preconditioner form 𝑃𝐴𝑥=𝑃𝑏, (3) where 𝑃 is called the preconditioner or a preconditioning matrix. several preconditioned iterative methods have been proposed [1–6]. Since 𝑃 is nonsingular, (1) and (3) have the same solutions. We are considering here systems with a unique solution. It is well known that system (3) can be transformed by an iteration as follows: 𝑥^𝑘+1=(𝐼−𝑃𝐴)𝑥^𝑘+𝑃𝑏. (4) This iteration is called the Richardson iteration for preconditioning system (3). In this paper, we consider the iteration methods by the following form: 𝑥^𝑘+1=𝑇𝑥^𝑘+𝑓, (5) where 𝑇 represents the iteration matrix.

Lemma 1 (see [7, 8]). For the iteration formula (5) to produce a sequence converging to (𝐼−𝐵)⁻¹𝑏, for any starting point 𝑥⁰, it is necessary and sufficient that the spectral radius of 𝐵 be less than one.

We will use the following notations. A matrix 𝐴=(𝑎_𝑖𝑗) is called a row diagonally dominant if _𝑖≠𝑗𝑎_𝑖𝑗<𝑎_𝑖𝑖for𝑖=1,…,𝑛, (6) and column diagonally dominant if _𝑖≠𝑗𝑎_𝑖𝑗<𝑎_𝑗𝑗for𝑗=1,…,𝑛. (7) The Frobenius inner product of 𝐴 and 𝐵 is defined by ⟨𝐴,𝐵⟩_𝐹=trace𝐴^𝑇𝐵, (8) where trace (𝐴) denotes the trace of a matrix 𝐴, and 𝐴^𝑇 stands for the transpose of 𝐴, and the spectral radius of a matrix 𝐴 is denoted by 𝜌(𝐴). Let 𝐴 be decomposed as 𝐴=𝐷−𝐿−𝑈 in which 𝐷 is the diagonal of 𝐴, 𝐿 is the strict lower part of 𝐴, and 𝑈 is the strict upper part of 𝐴.

Using the Frobenius norm, the preconditioning matrix in [6] is given by 𝑃_𝑓=𝑎₁₁𝑎₁²₂𝑎₂₂𝑎₂²₂⋱𝑎_𝑛𝑛𝑎_𝑛²₂, (9) where 𝑎_𝑖 stands for the 𝑖th row of the matrix 𝐴.

The second preconditioning matrix is 𝑃₁=𝛼𝐼 and 𝛼 is computed to minimize the infinity norm of the iteration matrix.

The small gap for a matrix 𝐴 is defined by sg(𝐴)=min_{𝑗=1,…,𝑛}𝑎_𝑗𝑗−^𝑛_𝑖≠𝑗𝑎_𝑖𝑗. (10) Obviously, sg(𝐴) is positive for diagonally dominant matrices. we can also suppose the diagonal entries are positive, else, it is true by multiplying the corresponding rows with −1.

Then, the preconditioner obtained by minimizing the infinity norm is given by 2𝛼=‖𝐴‖_∞(𝐴).+sg (11) We can easily find that the diagonal preconditioner is constant diagonal. Using the idea in [9], we can obtain the iterations associated with the preconditioners 𝑃_𝑓 and 𝑃_∞ as follows.

Solving the iteration 𝑥^𝑘+1=𝐼−𝜔𝑃_𝑓𝐴𝑥^𝑘+𝜔𝑃_𝑓𝑏, (12) the matrix 𝐴 is decomposed by 𝐴=𝐷+𝐿+𝑈 and 𝜔𝑃_𝑓𝐴 is moved to the left-hand side, we obtain 𝐼+𝜔𝑃_𝑓𝐿𝑥^𝑘+1=𝐼−𝜔𝑃_𝑓(𝐷+𝑈)𝑥^𝑘+𝜔𝑃_𝑓𝑏. (13) Because 𝐼+𝜔𝑃_𝑓𝐿 is inverse, we can easily get 𝑥^𝑘+1=𝐼−𝑃_𝑓⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃_𝑓⁻¹+𝜔𝐿⁻¹𝜔𝑏 (14) or 𝑥^𝑘+1=𝐼−𝜔𝐼+𝜔𝑃_𝑓𝐿⁻¹𝑃_𝑓𝐴𝑥^𝑘+𝐼+𝜔𝑃_𝑓𝐿⁻¹𝜔𝑃_𝑓𝑏. (15) This iteration is considered as sequential Frobenius norm iteration.

Similarly, we obtained and built the infinity norm iteration associated with 𝑃_∞ as follows: 𝑥^𝑘+1=𝐼−𝑃_∞⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃_∞⁻¹+𝜔𝐿⁻¹𝜔𝑏 (16) or 𝑥^𝑘+1=𝐼−𝜔𝐼+𝜔𝑃_∞𝐿⁻¹𝑃_∞𝐴𝑥^𝑘+𝐼+𝜔𝑃_∞𝐿⁻¹𝜔𝑃_∞𝑏. (17) Now, we have set two preconditioned SOR iterative methods which use 𝑃_𝑓 and 𝑃_∞ as a preconditioner.

In this paper, first in Section 2, we discuss the convergence of the preconditioned SOR iterative method which uses 𝑃_𝑓 and 𝑃_∞ as a preconditioner. In Section 3, we provide numerical experiments to illustrate the theoretical results obtained in Section 2, and we find if we choose the set of parameters; then our method has smaller spectral radii of the iterative matrices than the SOR method, which is shown through numerical examples.

2. Main Results

2.1. The Sequential Frobenius Norm Iteration

Theorem 2. Suppose 1≤𝜔<2 and 𝐴 is positive definite; then the iteration 𝑥^𝑘+1=𝐼−𝑃_𝑓⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃_𝑓⁻¹+𝜔𝐿⁻¹𝜔𝑏, (18) converges for any starting point 𝑥⁰.

Proof. By simple calculation, the iteration matrix can be written as 𝑃_𝑓⁻¹+𝜔𝐿⁻¹𝑃_𝑓⁻¹−(𝐷+𝑈)+(𝜔−1)𝐿, (19) let 𝜆 be corresponding eigenvalue; then 𝑃_𝑓⁻¹−(𝐷+𝑈)+(𝜔−1)𝐿𝑥=𝜆𝑃_𝑓⁻¹+𝜔𝐿𝑥 (20) or 𝑃_𝑓⁻¹−𝐴+𝐿+(𝜔−1)𝐿𝑥=𝜆𝑃_𝑓⁻¹+𝜔𝐿𝑥, (21) we get that −𝐴𝑥=(𝜆−1)𝑃_𝑓⁻¹+𝜔𝐿𝑥, (22) then −𝑥^∗𝐴𝑥=(𝜆−1)𝑥^∗𝑃_𝑓⁻¹+𝜔𝐿𝑥. (23) The following proof is similar in [9], so we omit it; then the theorem is obtained.

2.2. The Sequential Infinity Norm Iteration

Theorem 3. Suppose 1≤𝜔<2 and 𝐴 is positive definite. If every diagonal entry of 𝐴 satisfies ‖𝐴‖_∞+sg(𝐴)>𝜔𝑎_𝑖𝑖, (24) then the iteration 𝑥^𝑘+1=𝐼−𝑃_∞⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃_∞⁻¹+𝜔𝐿⁻¹𝜔𝑏, (25) converges for any starting point 𝑥⁰.

Proof. The proof of this theorem is similar to the previous one. We notice the diagonal entries of the matrix 2𝑃_∞⁻¹−𝜔𝐷 are ‖𝐴‖_∞+sg(𝐴)−𝜔𝑎_𝑖𝑖, (26) by the assumption ‖𝐴‖_∞+sg(𝐴)>𝜔𝑎_𝑖𝑖, so the diagonal entries of matrix 2𝑃_∞⁻¹−𝜔𝐷 are positive which completes the proof.

Now, we modify the infinity norm preconditioner for diagonally dominant matrices. Since the eigenvalues of a positive definite matrix 𝐴 lie in the interval (0,𝜔‖𝐴‖_∞+𝜖) for arbitrarily small number 𝜖, the preconditioning matrix 𝑃 is defined by 𝑃_∞=2𝜔‖𝐴‖_∞,+𝜖 (27) for any 𝜖>0. Especially, 𝜖=0 if 𝜌(𝐴)<‖𝐴‖_∞.

Theorem 4. For any 𝜖>0, if 𝐴 is positive definite; then the iteration 𝑥^𝑘+1=𝐼−𝑃_∞⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃_∞⁻¹+𝜔𝐿⁻¹𝜔𝑏, (28) converges for any starting point 𝑥⁰, where 1≤𝜔<2.

Proof. The proof is similar to Theorem 2, we notice the condition of 2𝑃_∞⁻¹−𝜔𝐷+𝜔𝐴≻𝐴 (29) or 2𝑃_∞⁻¹−𝜔𝐷≻0, and the diagonal entries are 2𝜔‖𝐴‖_∞+𝜖2−𝜔𝑎_𝑖𝑖=𝜔‖𝐴‖_∞+𝜖−𝜔𝑎_𝑖𝑖>0. (30) Hence, we obtain this result.

Next, we will obtain a general result for previously positive definite preconditioners.

Theorem 5. Suppose positive definite matrices of 𝐴 and 𝑃 satisfy 2𝑃⁻¹−𝜔𝐷≻0, (31) where 𝐷 is the diagonal of 𝐴 and 1≤𝜔<2; then the iteration 𝑥^𝑘+1=𝐼−𝑃⁻¹+𝜔𝐿⁻¹𝐴𝑥^𝑘+𝑃⁻¹+𝜔𝐿⁻¹𝜔𝑏, (32) converges for any starting point 𝑥⁰.

Remark 6. In Theorem 5, the matrix 𝑃 just satisfies the condition needed in the proof of Theorem 2; hence, this theorem is a general result for previous preconditioners.

3. Numerical Experiments

In this section, we provide numerical experiments to illustrate the theoretical results obtained in Section 2. All numerical experiments are carried out using MATLAB 7.1. The spectral radii of various iteration matrices are shown in Figures 1 and 2. For simplicity of comparison, suppose that all of 𝜔=1.9. Let (𝑜) denote the spectral radii of the SOR iteration matrices, let (⋅) denote the spectral radii of the Frobenius norm preconditioner iteration matrices, let (∗) denote the spectral radii of the infinity norm preconditioner, and let (∇) denote the spectral radii of the infinity-𝜖 preconditioner with 𝜖=0.1.

(a)

(b)

(c)

Figure 1 

Spectral radius for random PSD matrices 𝐴𝑇𝐴+diag(rand(𝑛,1)) of size 𝑛=50 (a), 𝐴𝑇𝐴+10∗diag(rand(𝑛,1)) of size 𝑛=50 (b) and 𝐴𝑇𝐴+30∗diag(rand(𝑛,1)) (c) of size 𝑛=150.

Figure 2 

Spectral radius of random PSD matrices 𝐴𝑇𝐴+10∗diag(rand(𝑛,1)) with nearly half of the entries of 𝐴 negative and 𝑛=50.

Remark 7. The previous numerical experiments indicate that the spectral radii of iterative matrices with three proposed preconditioners achieve significant improvement over the spectral radii of SOR iterative matrices.

Remark 8. As the proportion of negative to positive entries is increased, the spectral radius of random positive PSD matrices 𝐴^𝑇𝐴+10∗diag(rand(𝑛,1)) with half of the entries of 𝐴 negative with infinity norm preconditioner (∗) becomes larger than 1. But both of infinity norm preconditioners have faster convergence rates than SOR method.

Remark 9. Here, we maintain both of infinity norm iterations because there are cases when 𝑃_∞=(2/(‖𝐴‖_∞+sg(𝐴)))(∗) works better than 𝑃_∞=(2/(𝜔‖𝐴‖_∞+𝜖))(∇).

Acknowledgments

The authors would like to thank the anonymous referees for their helpful comments and advice, which greatly improved the paper. The study was financially supported by National Natural Science Foundation (nos. 11161041, 71301111), Fundamental Research Funds for the Central Universities (31920130005).