Modified Preconditioned GAOR Methods for Systems of Linear Equations

Abstract

Three kinds of preconditioners are proposed to accelerate the generalized AOR (GAOR) method for the linear system from the generalized least squares problem. The convergence and comparison results are obtained. The comparison results show that the convergence rate of the preconditioned generalized AOR (PGAOR) methods is better than that of the original GAOR methods. Finally, some numerical results are reported to confirm the validity of the proposed methods.

1. Introduction

Consider the generalized least squares problem min_{𝑥∈ℝ𝑛}(𝐴𝑥−𝑏)^𝑇𝑊⁻¹(𝐴𝑥−𝑏), (1) where 𝑥∈ℝ^𝑛, 𝐴∈ℝ^𝑛×𝑛, 𝑏∈ℝ^𝑛, and the variance-covariance matrix 𝑊∈ℝ^𝑛×𝑛 is a known symmetric and positive-definite matrix. This problem has many scientific applications and one of the applications is a parameter estimation in mathematical model [1, 2].

In order to solve the problem simply, one has to solve a linear system of the equivalent form as follows: 𝐹𝑦=𝑓, (2) where 𝐹=𝐼−𝐵𝐻𝐾𝐼−𝐶, (3) with 𝐵∈ℝ^𝑝×𝑝, 𝐶∈ℝ^𝑞×𝑞, and 𝑝+𝑞=𝑛. Without loss of generality, we assume that 𝐹=ℐ−ℒ−𝒰, where ℐ is the identity matrix, and ℒ and 𝒰 are strictly lower and upper triangular matrices obtained from 𝐹, respectively. So we can pretty easily get that ℐ=𝐼00𝐼,ℒ=00−𝐾0,𝒰=𝐵−𝐻0𝐶. (4)

In order to get the approximate solutions of the linear system (2), a lot of iterative methods such as Jacobi, Gauss-Seidel (GS), successive over relaxation (SOR), and accelerated over relaxation (AOR) have been studied by many authors [3–8]. These iterative methods have very good results, but have a serious drawback because of computing the inverses of 𝐼−𝐵 and 𝐼−𝐶 in (3). To avoid this drawback, Darvishi and Hessari [9] proposed the generalized convergence of the generalized AOR (GAOR) method when the coefficient matrix 𝐹 is a diagonally dominant matrix. The GAOR method [10, 11] can be defined as follows: 𝑦_𝑘+1=𝒯_𝛾𝜔𝑦_𝑘+𝜔𝑔,𝑘=0,1,2,…, (5) where 𝒯_𝛾𝜔=𝐼0𝛾𝐾𝐼⁻¹×(1−𝜔)𝐼+𝜔−𝛾,00−𝐾0+𝜔𝐵−𝐻0𝐶𝑔=𝐼0−𝛾𝐾𝐼𝑓. (6) Here, 𝜔 and 𝛾 are real parameters with 𝜔≠0. The iteration matrix is rewritten briefly as 𝒯_𝛾𝜔=()𝜔()1−𝜔𝐼+𝜔𝐵−𝜔𝐻𝛾−1𝐾−𝛾𝜔𝐾𝐵1−𝜔𝐼+𝜔𝐶+𝑤𝛾𝐾𝐻. (7)

To improve the convergence rate of the GAOR iterative method, a preconditioner should be applied. Now we can transform the original linear system (2) into the preconditioned linear system 𝑃𝐹𝑦=𝑃𝑓, (8) where 𝑃 is the preconditioner. 𝑃𝐹 can be expressed as 𝑃𝐹=𝐼−𝐵^∗𝐻^∗𝐾^∗𝐼−𝐶^∗. (9) Meanwhile, the PGAOR method for solving the preconditioned linear system (8) is defined by 𝑦_𝑘+1=𝒯^∗_𝛾𝜔𝑦_𝑘+𝑤𝑔^∗,𝑘=0,1,2,…, (10) where𝒯^∗_𝛾𝜔=(1−𝜔)𝐼+𝜔𝐵^∗−𝜔𝐻^∗𝜔𝛾−1𝐾^∗−𝛾𝜔𝐾^∗𝐵^∗(1−𝜔)𝐼+𝜔𝐶^∗+𝛾𝜔𝐾^∗𝐻^∗,𝑔=𝐼0−𝛾𝐾^∗𝐼𝑃𝑓. (11)

In this paper, we propose three new types of preconditioners and study the convergence rate of the preconditioned GAOR methods for solving the linear system (2). This paper is organized as follows. In Section 2, some notations, definitions, and preliminary results are presented. In Section 3, three new types of preconditioners are proposed and compared with that of the original GAOR methods. Lastly, a numerical example is provided to confirm the theoretical results studied in Section 4.

2. Preliminaries

For vector 𝑥∈ℝ^𝑛, 𝑥≥0(𝑥>0) denotes that all components of 𝑥 are nonnegative (positive). For two vectors 𝑥,𝑦∈ℝ^𝑛, 𝑥≥𝑦(𝑥>𝑦) means that 𝑥−𝑦≥0(𝑥−𝑦>0). These definitions are carried immediately over to matrices. A matrix 𝐴 is said to be irreducible if the directed graph of 𝐴 is strongly connected. 𝜌(𝐴) denotes the spectral radius of 𝐴. Some useful results are provided as follows.

Lemma 1 (see [7]). Let 𝐴≥0 be an irreducible matrix. Then,(a)𝐴has a positive eigenvalue equal to its spectral radius. (b)𝐴 has an eigenvector 𝑥>0 corresponding to 𝜌(𝐴). (c)𝜌(𝐴) is a simple eigenvalue of 𝐴.

Lemma 2 (see [12]). Let 𝐴 be a nonnegative matrix. Then,(i)if 𝛼𝑥≤𝐴𝑥 for some nonnegative vector 𝑥, 𝑥≠0, then 𝛼≤𝜌(𝐴). (ii)If 𝐴𝑥≤𝛽𝑥 for some positive vector 𝑥, then 𝜌(𝐴)≤𝛽. Moreover, if 𝐴 is irreducible and if 0≠𝛼𝑥≤𝐴𝑥≤𝛽𝑥 for some nonnegative vector 𝑥, then 𝛼≤𝜌(𝐴)≤𝛽 and 𝑥 is a positive vector.

3. Preconditioned GAOR Methods

To solve the linear system (2) with the coefficient matrix 𝐹 in (3), we consider the preconditioners as follows: 𝑃_𝑖=𝐼+𝑆𝑆=𝐼0_𝑖𝐼,𝑖=1,2,3, (12) where 𝑆₁=00⋯−𝑘_1𝑝00⋯−𝑘_2𝑝0⋱⋮00⋯−𝑘_𝑞𝑝,𝑆₂=0−𝑘₁₂⋯00−𝑘₂₁0−𝑘₂₃⋯00−𝑘₃₂⋱⋮⋮⋮⋮⋯0−𝑘_{𝑞−1,𝑝}00⋯−𝑘_{𝑞,𝑝−1}0,𝑆₃=0−𝑘₁₂⋯000⋱⋮0⋮0−𝑘_{𝑞−1,𝑝}−𝑘_𝑞1𝛼,().…00𝛼>0 (13) The preconditioned coefficient matrix 𝑃_𝑖𝐹 can be expressed as 𝑃_𝑖𝐹=𝐼−𝐵𝐻𝐾+𝑆_𝑖(𝐼−𝐵)𝑆_𝑖𝐻+𝐼−𝐶,𝑖=1,2,3, (14) where𝐾+𝑆₁()=𝑘𝐼−𝐵¹¹𝑘₁₂⋯𝑘_1𝑝𝑏_𝑝𝑝𝑘₂₁𝑘₂₂⋯𝑘_2𝑝𝑏_𝑝𝑝𝑘_𝑞1𝑘_𝑞2⋯𝑘_𝑞𝑝𝑏_𝑝𝑝,𝐾+𝑆₂()=𝑘𝐼−𝐵¹¹𝑘₁₂𝑏₂₂⋯𝑘_1,𝑝−1𝑘_1𝑝𝑘₂₁𝑏₁₁+𝑘₂₃𝑏₃₁𝑘₂₂𝑘₂₁𝑏₁₃+𝑘₂₃𝑏₃₃⋯𝑘_2𝑝𝑘₃₁𝑘₃₂𝑏₂₂+𝑘₃₄𝑏₄₂⋱⋮⋮⋮⋮⋯0𝑘_{𝑞−1,𝑝−2}𝑏_{𝑞−2,𝑝}+𝑘_{𝑞−1,𝑝}𝑏_𝑞𝑝𝑘_𝑞1𝑘_𝑞2⋯𝑘_{𝑞,𝑝−1}𝑏_{𝑞−1,𝑝−1}𝑘_𝑞𝑝,𝐾+𝑆₃()=𝑘𝐼−𝐵¹¹𝑘₁₂𝑏₂₂⋯𝑘_1𝑝𝑘₂₁𝑘₂₂⋱𝑘_2𝑝⋮⋮…𝑘_{𝑞−1,𝑝}𝑏_𝑝𝑝𝑘_𝑞11−1−𝑏₁₁𝛼𝑘_𝑞2⋯𝑘_𝑞𝑝. (15)Based on the discussed above, 𝑃_𝑖𝐹 can be spitted as 𝑃_𝑖𝐹=ℐ−ℒ_𝑖−𝒰_𝑖,𝑖=1,2,3. (16) Similarly, ℐ=𝐼00𝐼,ℒ_𝑖=00−𝐾−𝑆_𝑖()0,𝒰𝐼−𝐵_𝑖=𝐵−𝐻0𝐶−𝑆_𝑖𝐻,𝑖=1,2,3. (17) The preconditioned GAOR methods for solving 𝑃_𝑖𝐹𝑦=𝑃_𝑖𝑓 are defined by 𝑦_𝑘+1=𝒯^∗_𝛾𝜔𝑖𝑦_𝑘+𝜔𝑔^∗_𝑖,𝑘=0,1,2,…,𝑖=1,2,3, (18) where 𝜔𝑔^∗_𝑖=ℐ−Γℒ_𝑖⁻¹Ω𝑃_𝑖𝑓,𝑖=1,2,3, (19) with 𝒯^∗_𝛾𝜔𝑖=𝐼−Γℒ_𝑖⁻¹𝐼−Ω+(Ω−Γ)ℒ_𝑖+Ω𝒰_𝑖,𝑖=1,2,3, (20) where 𝜔Ω=¹𝐼00𝜔₂𝛾𝐼,,Γ=¹𝐼00𝛾₂𝐼. (21) For 𝑖=1,2,3, we have𝒯^∗_𝛾𝜔𝑖=1−𝜔₁𝐼+𝜔₁𝐵−𝜔₁𝐻𝜔₁𝛾₂−𝜔₂𝐾+𝑆_𝑖(𝐼−𝐵)1−𝛾₂𝐼+𝛾₂𝐶−𝛾₂𝑆_𝑖𝐻−𝜔₁𝛾₂𝐾+𝑆_𝑖(𝐼−𝐵)𝐵+𝜔₁𝛾₂𝐾+𝑆_𝑖(𝐼−𝐵)𝐻. (22)

Next, we will study the convergence condition of the PGAOR methods. For simplicity, without loss of generality, we can assume that 𝐻≤0,𝐾≤0,𝐵≥0,𝐶≥0,0<𝜔₁≤1,0<𝜔₂≤1,0<𝛾₂≤𝜔₂𝜔₁. (23) Then, we have the following theorem.

Theorem 3. Let 𝒯_𝛾𝜔 and 𝒯^∗_𝛾𝜔1 be the iteration matrices of the GAOR method and the PGAOR method corresponding to problem (2), which are defined by (7) and (22), respectively. If matrix 𝐹 in (3) is an irreducible matrix then it holds that 𝜌(𝒯_𝛾𝜔)≠1, 𝜌𝒯^∗_𝛾𝜔1>𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔𝜌𝒯>1,_∗𝛾𝜔1<𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔<1. (24)

Proof. By some simple calculations on (7), one can get 𝒯_𝛾𝜔=1−𝜔₁𝐼+𝜔₁𝐵−𝜔₁𝐻𝜔₁𝛾₂−𝜔₂𝐾1−𝜔₂𝐼+𝜔₂𝐶+𝜔₁𝛾₂.00−𝐾𝐵𝐾𝐻 (25) Since here 𝐹 is irreducible, one can pretty easily obtain that 𝒯_𝛾𝜔 is nonnegative and irreducible by the above assumptions. And so on, one can also easily prove that 𝒯^∗_𝛾𝜔1 is non-negative and irreducible. By Lemma 1, there exists a positive vector 𝑥>0 such that 𝒯_𝛾𝜔𝑥=𝜆𝑥, (26) where 𝜆=𝜌(𝒯_𝛾𝜔).
One can easily have [ℐ−Ω+(Ω−Γ)ℒ+Ω𝒰]𝑥=𝜆(ℐ−Γℒ)𝑥. (27) That is, (ℐ−Ω)𝑥=𝜆(ℐ−Γℒ𝑥−(Ω−Γ)ℒ𝑥−Ω𝒰𝑥). (28) With the same vector 𝑥>0, it holds 𝒯^∗_𝛾𝜔1𝑥−𝜆𝑥=ℐ−Γℒ₁⁻¹×ℐ−Ω+(Ω−Γ)ℒ₁−𝜆ℐ−Γℒ₁𝑥. (29) Using (22), (26), and (28), we can obtain 𝒯^∗_𝛾𝜔1𝑥−𝜆𝑥=ℐ−Γℒ₁⁻¹×(Ω−Γ)ℒ₁𝒰−ℒ+Ω₁−𝒰+𝜆Γℒ₁𝑥=−ℒℐ−Γℒ₁⁻¹Ω𝒰₁−𝒰+ℒ₁+()Γℒ−ℒ𝜆−1₁𝑥=−ℒℐ−Γℒ₁⁻¹Ω𝒰₁−𝒰+ℒ₁𝑥+()−ℒ𝜆−1ℐ−Γℒ₁⁻¹Γℒ₁𝑥=−ℒℐ−Γℒ₁⁻¹00−𝜔₂𝑆₁(𝐼−𝐵)−𝜔₂𝑆₁𝐻𝑥+(𝜆−1)ℐ−Γℒ₁⁻¹×00−𝛾₂𝑆₁(𝐼−𝐵)0𝑥. (30) Meanwhile, we have ℐ−Γℒ₁⁻¹00−𝜔₂𝑆₁(𝐼−𝐵)−𝜔₂𝑆₁𝐻𝑥=ℐ−Γℒ₁⁻¹𝜔00₂𝜔₁𝑆₁0−𝜔₁𝐼+𝜔₁𝐵−𝜔₁𝐻𝑥=00ℐ−Γℒ₁⁻¹𝜔00₂𝜔₁𝑆₁0×−𝜔₁𝐼+𝜔₁𝐵−𝜔₁𝐻𝜔₁𝛾₂𝐾−𝜔₁𝛾₂𝐾𝐵−𝜔₂𝐼+𝜔₂𝐶+𝜔₁𝛾₂𝑥=𝐾𝐻ℐ−Γℒ₁⁻¹𝜔00₂𝜔₁𝑆₁0𝒯_𝑤𝑟𝑥=()−ℐ𝜆−1ℐ−Γℒ₁⁻¹𝜔00₂𝜔₁𝑆₁0𝑥. (31) By far, we can easily get 𝒯^∗_𝛾𝜔1𝑥−𝜆𝑥=(𝜆−1)ℐ−Γℒ₁⁻¹×𝜔00₂𝜔₁𝑆₁0+00−𝛾₂𝑆₁()0𝑥=()𝐼−𝐵𝜆−1𝐼0−𝛾₂𝐾+𝑆₁(𝐼×𝜔𝐼−𝐵)00₂𝜔₁−𝛾₂𝑆₁+𝛾₂𝑆₁𝑥=()𝜔𝐵0𝜆−100₂𝜔₁−𝛾₂𝑆₁+𝛾₂𝑆₁𝐵0𝑥. (32) In view of the abovementioned assumptions, we have that 𝜔00₂𝜔₁−𝛾₂𝑆₁+𝛾₂𝑆₁𝐵0𝑥>0. (33) Then, if 𝜆=𝜌(𝒯_𝑤𝑟)>1, then 𝒯^∗_𝛾𝜔1−𝜆𝑥>0,𝒯^∗_𝛾𝜔1−𝜆𝑥≠0. (34) From Lemma 2, we can easily get 𝜌𝒯^∗_𝛾𝜔1>𝜌𝒯_𝑤𝑟>1. (35) Similarly, if 𝜆=𝜌(𝒯_𝑤𝑟)<1, then 𝒯^∗_𝛾𝜔1−𝜆𝑥<0,𝒯^∗_𝛾𝜔1−𝜆𝑥≠0. (36) So we have 𝜌𝒯^∗_𝛾𝜔1<𝜌𝒯_𝛾𝜔<1. (37) If 𝜆=𝜌(𝒯_𝛾𝜔)=1, then we may get that 𝐹𝑦=𝑓 but 𝑦≠0, which is contradictory to the fact of nonsingular matrix 𝐹 by assumptions; this completes the conclusion of the theorem.

Theorem 4. Let 𝒯_𝛾𝜔 and 𝒯^∗_𝛾𝜔2 be the iteration matrices of the GAOR method and the PGAOR method corresponding to problem (2), which are defined by (7) and (22), respectively. If the matrix 𝐹 in (3) is an irreducible matrix satisfying 𝑏₁₁>0,𝑘_𝑖𝑗≠0𝑓𝑜𝑟𝑖−𝑗=1, (38) then it holds that 𝜌(𝑇_𝑤𝑟)≠0, 𝜌𝒯^∗_𝛾𝜔2>𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔𝜌𝒯>1,_∗𝛾𝜔2<𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔<1. (39)

Proof. One can easily prove this theorem by using similar arguments of Theorem 3.

Similarly, we have the following theorem.

Theorem 5. Let 𝒯_𝛾𝜔 and 𝒯^∗_𝛾𝜔3 be the iteration matrices of the GAOR method and the PGAOR method corresponding to problem (2), which are defined by (7) and (22), respectively. If the matrix 𝐹 in (3) is an irreducible matrix satisfying 𝛼>1,𝑘_𝑞1<0,𝑏₁₁𝑘>0,_𝑖,𝑖+1<0𝑓𝑜𝑟𝑖=1,2,…,𝑞−1, (40) then it holds that 𝜌(𝒯_𝛾𝜔)≠1, 𝜌𝒯^∗_𝛾𝜔3>𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔𝜌𝒯>1,_∗𝛾𝜔3<𝜌𝒯_𝛾𝜔≠1𝑖𝑓𝜌𝒯_𝛾𝜔<1. (41)

4. Numerical Examples

In this section, we give numerical examples to demonstrate the conclusions drawn above. The numerical experiments were done by using MATLAB 7.0.

Example 1. Consider the following Laplace equation: 𝜕²𝑢𝑥,𝑦𝜕𝑥²+𝜕²𝑢𝑥,𝑦𝜕𝑦²=0. (42) Under a uniform square domain, applying the five-point finite difference method with the uniform mesh size, we can get the following linear system: ℱ𝑥=𝑓, (43) where1ℱ=21000000−810−8−18012100000−810000021000−8−18−18−18010002100−8−181000000021000−8−18010000021000−8−18100−8−181002−100008−18−18−1810002100000−810−810002−100810−810−8−18100020−1810000−8100002. (44)The coefficient matrix ℱ is spitted as ℱ=𝐼−𝐵𝐻𝐾𝐼−𝐶, (45) where 1𝐵=20100000210000002100000021000000201000002,1𝐶=20100002100000210000020100002,1𝐻=0−810−8−1810−8−10008−18−18−180−18−18100000−8−1801000−8−18,1𝐾=00−8−18−1008−18−18−1810000−810−80−1810−810−8−18−1810000−8. (46)

Table 1 reveals the spectral radii of the GAOR methods and the PGAOR methods. It tells that the spectral radii of the preconditioned PGAOR methods are smaller than those of the GAOR methods, so we can get that the proposed three preconditioners can accelerate the speed rate of the GAOR method for the linear systems (2). The results in Table 1 are in accordance with Theorems 3–5.

Table 1 

Spectral radii of GAOR method and PGAOR method.

Example 2. The coefficient matrix 𝐹 in (3) is given by 𝐹=𝐼−𝐵𝐻𝐾𝐼−𝐶, (47) where 𝐵=(𝑏_𝑖𝑗)_𝑝×𝑝, 𝐶=(𝑐_𝑖𝑗)_𝑞×𝑞, and 𝑝+𝑞=𝑛, 𝐻=(ℎ_𝑖𝑗)_𝑝×𝑞, 𝐾=(𝑘_𝑖𝑗)_𝑞×𝑝 with 𝑏_𝑖𝑖=1𝑏𝑖+1,𝑖=1,…𝑝,_𝑖𝑗=1−130𝑏30𝑗+𝑖,𝑗>𝑖,𝑖=1,…𝑝−1,𝑗=2,…𝑝,_𝑖𝑗=1−130,𝑐3010𝑖−𝑗+1+𝑖𝑖>𝑗,𝑖=2,…𝑝,𝑗=1,…𝑝−1,_𝑖𝑖=1𝑐𝑛+𝑖+1,𝑖=1,…𝑞,_𝑖𝑗=1−130,𝑐30𝑛+𝑗+𝑛+𝑖𝑗>𝑖,𝑖=1,…,𝑞−1,𝑗=2,…,𝑞,_𝑖𝑗=1−130,𝑘30𝑖−𝑗+1+𝑛+𝑖𝑖>𝑗,𝑖=2,…,𝑞,𝑗=1,…𝑞−1,_𝑖𝑗=1−130𝑛+𝑖−𝑗+1+𝑛+𝑖,ℎ30𝑖=1,…,𝑞,𝑗=1,…,𝑝,_𝑖𝑗=1−130𝑛+𝑗+𝑖30,𝑖=1,…,𝑝,𝑗=1,…,𝑞. (48)

Obviously, 𝐹 is irreducible. Table 2 shows the spectral radii of the corresponding iteration matrices with 𝑛=8 and 𝑚=6.

Table 2 

Spectral radii of GAOR method and PGAOR method.

Similarly, in Table 2, we get that the results are in concord with Theorems 3–5.

Acknowledgments

The authors would like to thank the anonymous referees for their helpful suggestions, which greatly improved the paper. This research of this author was supported by NSFC Tianyuan Mathematics Youth Fund (no. 11026040), by Science and Technology Development Plan of Henan Province (no. 122300410316), and by Natural Science Foundations of Henan Province (no. 13A110022).