Convergence Results on Iteration Algorithms to Linear Systems

Abstract

In order to solve the large scale linear systems, backward and Jacobi iteration algorithms are employed. The convergence is the most important issue. In this paper, a unified backward iterative matrix is proposed. It shows that some well-known iterative algorithms can be deduced with it. The most important result is that the convergence results have been proved. Firstly, the spectral radius of the Jacobi iterative matrix is positive and the one of backward iterative matrix is strongly positive (lager than a positive constant). Secondly, the mentioned two iterations have the same convergence results (convergence or divergence simultaneously). Finally, some numerical experiments show that the proposed algorithms are correct and have the merit of backward methods.

1. Introduction

The primal goal of this paper is to study the iterative methods of the linear systems: 𝐴𝑥=𝑏, (1) where 𝐴 is a given 𝑛×𝑛 complex or real matrix.

It is well known that linear systems arise in studies in many areas such as engineering and industrial science. For example, in the field of numerical solutions of differential-algebraic equations (DAE) and ordinary differential equations (ODE) [1–3] it is very important to solve (1). In digital image and signal processing, especially in compressed sensing, Stojnic [4] has mentioned that the systems (1) are the mathematical background of compressed sensing problems and studied sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations for the case of the so-called block-sparse unknown vectors. In the blind source separation of signal, Congedo et al. [5] have showed that it is very important to solve (1) and proposed a special method with joint singular value decomposition. In the field of biomedical engineering, Deo et al. [6] have mentioned that the cardiac electrical activity can be described by the bidomain equations and pointed out that the numerical solution of partial differential equations (PDEs) associated with bidomain problems often leads to (1). Moreover, they have proposed a novel preconditioner for the PCG method to solve (1) and a cheap iterative method such as successive overrelaxation (SOR) to further refine the solution for a desired accuracy. In 2008, Shou et al. [7] have showed that the reconstruction of epicardial potentials (EPs) from body surface potentials (BSPs) can be characterized as an ill-posed inverse problem and geometric errors in the ECG inverse problem will directly affect the calculation of transfer matrix 𝐴 in (1). In the field of systems and control science, Ding and Chen [8] have pointed out that Sylvester equations in systems and control especially Lyapunov equations in continuous- and discrete-time stability analysis can be converted into equivalent equations as (1). In the field of machine learning many problems of classification and regression, such as single-hidden layer neural networks [9, 10], support vector machines, functional neural networks, and so on, can be summarized as (1). Therefore, the solution of (1) is very important in scientific computing.

The methods to solve linear systems can be roughly divided into two categories: direct methods and iterative methods 1. Iterative methods are more suitable than direct methods for large linear systems [11, 12]. The current research on iterative algorithms has been more mature, but how to make it fit the new architecture model is complicated. In order to gain more good performance, acceleration has been applied and architecture has been considered [13].

In this paper, we do some research with the iterative algorithm. To this end, the paper is organized as follows. In Section 2, we introduce the backward MPSD (backward modified preconditioned simultaneous displacement) iterative method which is a unified form of some important backward iterations. In Section 3, we first introduce some important lemmas which will be used and then we obtained the convergence results between backward MPSD iteration and Jacobi iteration. We also proposed convergence results between some backward iterations and Jacobi iteration in the corollaries. In Section 4, some examples and numerical experiments have been done to make sure of the correctness of results. Especially, we point out that the backward iteration is better than the original one in many cases just like Example 4.

2. A Unified Framework of Iteration Matrix and Algorithm

The basic idea to solve (1) is matrix splitting. If we let 𝐴=𝐷−𝐶_𝐿−𝐶_𝑈,𝐿=𝐷⁻¹𝐶_𝐿,𝑈=𝐷⁻¹𝐶_𝑈, (2) where 𝐷=diag(𝐴) is a diagonal matrix obtained from 𝐴 and nonsingular and 𝐶_𝐿 and 𝐶_𝑈 are strictly lower and upper triangular matrices obtained from 𝐴, (1) becomes the equivalent one: (𝐼−𝐿−𝑈)𝑥=𝐷⁻¹𝑏. (3) At this moment, 𝐷⁻¹𝐴=(𝐼−𝐿−𝑈).

The Jacobi iterative matrix is 𝐵=𝐿+𝑈=𝐼−𝐷⁻¹𝐴. (4) The MPSD (modified preconditioned simultaneous displacement) iterative method is studied in [14–17].

If 𝜏>0 is a real constant, obviously (3) is equivalent to 𝜏(𝐼−𝐿−𝑈)𝑥=𝜏𝐷⁻¹𝑏. (5)

At the same time, if 𝜔₁,𝜔₂ are real constants, we can obtain the following equivalent from (5): 𝐼−𝜔₁𝑈−𝜔₂𝐿+𝜔₁𝜔₂𝑥=𝑈𝐿𝐼−𝜔₁𝑈−𝜔₂𝐿+𝜔₁𝜔₂𝑥+()𝑈𝐿−𝜏𝐼+𝜏𝐿+𝜏𝑈𝑥+𝜏𝐷⁻¹𝑏. (6)

It is easy to verify that 𝐼−𝜔₁𝑈𝐼−𝜔₂𝐿𝑥=(1−𝜏)𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂𝑈𝐿𝑥+𝜏𝐷⁻¹𝑏. (7)

With (7), we can construct the backward MPSD iterative method as follows: 𝑥^(𝑘+1)=𝑆_{𝜏,𝜔1,𝜔2}𝑥^(𝑘)+𝐼−𝜔₂𝐿⁻¹𝐼−𝜔₁𝑈⁻¹𝜏𝐷⁻¹𝑏, (8) where 𝑆_{𝜏,𝜔1,𝜔2}=𝐼−𝜔₂𝐿⁻¹𝐼−𝜔₁𝑈⁻¹×(1−𝜏)𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂𝑈𝐿, (9) which we named as backward MPSD iterative matrix.

Also, we have the following algorithm.

Backward MPSD Algorithm

Step  0 (Input). Matrix 𝐴, vector 𝑏, 𝜏>0, 𝜔₁, 𝜔₂, algorithm stop cutoff 𝜖.

Step  1 (Initialization). Compute 𝐷=diag(𝐴), 𝐶_𝐿, 𝐶_𝑈, 𝐿=𝐷⁻¹𝐶_𝐿,𝑈=𝐷⁻¹𝐶_𝑈,𝑥⁽⁰⁾=0, and set 𝑖∶=0.

Step  2. Compute matrix 𝑆_{𝜏,𝜔1,𝜔2} according to (9).

Step  3. Compute 𝑥^(𝑖+1) with (8).

Step  4. If ‖𝑥^(𝑖+1)−𝑥^(𝑖)‖²₂⩽𝜖, then stop and accept 𝑥^(𝑖) as the solution of (1); else 𝑖∶=𝑖+1; go to step 3.

Remark 1. With special values of 𝜔₁,𝜔₂, and 𝜏, we have the following.(1)When 𝜔₁=0, 𝜔₂=0, and 𝜏=1, we obtain the Jacobi iterative method.(2)When 𝜔₁=0, 𝜔₂=0, and 𝜏=𝜔, we obtain the backward JOR iterative method.(3)When 𝜔₁=1, 𝜔₂=0, and 𝜏=1, we obtain the backward G-S iterative method.(4)When 𝜔₁=𝜔, 𝜔₂=0, and 𝜏=𝜔, we obtain the backward SOR iterative method.(5)When 𝜔₁=𝜔, 𝜔₂=0, and 𝜏=𝛼, we obtain the backward AOR iterative method.(6)When 𝜔₁=𝜔, 𝜔₂=𝜔, and 𝜏=𝜔(2−𝜔), we obtain the backward SSOR iterative method.(7)When 𝜔₁=𝜔, 𝜔₂=𝜔, and 𝜏=𝜔, we obtain the backward EMA iterative method.(8)When 𝜔₁=𝜔, 𝜔₂=𝜔, and 𝜏=𝛼, we obtain the backward PSD iterative method.(9)When 𝜔₁=𝜔, 𝜔₂=𝜔, and 𝜏=1, we obtain the backward PJ iterative method.

The convergence relationship between the Gauss-Seidel iterative matrix and the Jacobi iterative matrix is studied in [12], and the generalized results are studied in [18]. Some eigenvalue relationships between other iterative matrices and Jacobi iterative matrix are studied with the p-cyclic case in [19–26]. Some backward iterations are studied in [27]. In the following we consider the convergence results between the backward MPSD iterative matrix and the Jacobi iterative matrix and obtain convergence relationships between some other backward iterative matrices and Jacobi matrix.

3. Convergence Results

In order to obtain the convergence results, we give some well-known results which will be used in the proof of Theorem 7 as follows.

Definition 2 (see [13]). The splitting 𝐴=𝑀−𝑁 with 𝐴 and 𝑀 nonsingular is called a regular splitting if 𝑀⁻¹≥0 and 𝑁≥0. It is called a weak regular splitting if 𝑀⁻¹≥0 and 𝑀⁻¹𝑁≥0.
It is obvious that a regular splitting is a weak regular splitting.

Lemma 3 (see [13]). The nonnegative matrix 𝑇∈𝑅^𝑛×𝑛 is convergent; that is, 𝜌(𝑇)<1 if and only if (𝐼−𝑇)⁻¹ exists and (𝐼−𝑇)⁻¹=∑^∞_𝑘=1𝑇^𝑘≥0.

Lemma 4 (see [13]). Let 𝐴=𝑀−𝑁 be a weak regular splitting of 𝐴, 𝐻=𝑀⁻¹𝑁. Then the following statements are equivalent. (1)𝐴⁻¹≥0; that is, 𝐴 is inverse-positive. (2)𝐴⁻¹𝑁≥0. (3)𝜌(𝐻)=𝜌(𝐴⁻¹𝑁)/(1+𝜌(𝐴⁻¹𝑁)) so that 𝜌(𝐻)<1.

Lemma 5 (see [12]). Let 𝐴≥0 be an irreducible 𝑛×𝑛 matrix. Then (1)𝐴 has a positive real eigenvalue equal to its spectral radius; (2)to 𝜌(𝐴), there corresponds an eigenvector 𝑥>0, (3)𝜌(𝐴) increases when any entry of 𝐴 increases, (4)𝜌(𝐴) is a simple eigenvalue of 𝐴.

Lemma 6 (see [12]). Let 𝐴=(𝑎_𝑖𝑗)≥0 be an irreducible 𝑛×𝑛 matrix. Then for any 𝑥>0, either min_{1≤𝑖≤𝑛}∑^𝑛_𝑗=1𝑎_𝑖𝑗𝑥_𝑗𝑥_𝑖<𝜌(𝐴)<max_{1≤𝑖≤𝑛}∑^𝑛_𝑗=1𝑎_𝑖𝑗𝑥_𝑗𝑥_𝑖 (10) or ∑^𝑛_𝑗=1𝑎_𝑖𝑗𝑥_𝑗𝑥_𝑖=𝜌(𝐴)∀𝑖. (11)

By the lemmas above, we give the convergence theorem in the following.

Theorem 7. Let the coefficient matrix 𝐴 of (1) be irreducible with 𝑎_𝑖𝑖≠0,∀𝑖, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_{𝜏,𝜔1,𝜔2} the backward MPSD iterative matrix. Then, for 0≤𝜔_𝑘<𝜏≤1, 𝑘=1,2, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1−𝜏. (2)One and only one of the following mutually exclusive relations is valid.(i)0<𝜌(𝐵)<1⇔1−𝜏<𝜌(𝑆_{𝜏,𝜔1,𝜔2})<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_{𝜏,𝜔1,𝜔2})=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1.

Thus, The Jacobi iterative method and the backward MPSD iterative method are either both convergent or both divergent.

Proof. Combining 𝜌(𝜔₂𝐿)=𝜌(𝜔₁𝑈)=0 with Lemma 3, we have (𝐼−𝜔₂𝐿)⁻¹≥0,(𝐼−𝜔₁𝑈)⁻¹≥0, and 𝑆_{𝜏,𝜔1,𝜔2}=𝐼−𝜔₂𝐿⁻¹𝐼−𝜔₁𝑈⁻¹×(1−𝜏)𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂=𝑈𝐿𝐼+𝜔₂𝐿+𝜔²₂𝐿²×+⋯𝐼+𝜔₁𝑈+𝜔²₁𝑈²×)+⋯(1−𝜏𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂≥()𝑈𝐿1−𝜏𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿. (12)
Since 𝑎_𝑖𝑖≠0 and 𝐴 is irreducible, 𝐼−𝐿−𝑈=𝐷⁻¹𝐴 and 𝐵=𝐿+𝑈 are irreducible. By 0≤𝜔_𝑘<𝜏≤1,𝑘=1,2, we have (1−𝜏)𝐼+(𝜏−𝜔₁)𝑈+(𝜏−𝜔₂)𝐿≥0 and irreducible. Thus, by (12), 𝑆_{𝜏,𝜔1,𝜔2}≥0 and is irreducible. By Lemma 5, there exists 𝜆=𝜌(𝑆_{𝜏,𝜔1,𝜔2})>0 and corresponding vector 𝑥=(𝑥₁,𝑥₂,…,𝑥_𝑛)^𝑇>0, such that 𝑆_{𝜏,𝜔1,𝜔2}𝑥=𝜆𝑥; namely, (1−𝜏)𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂𝑈𝐿𝑥=𝜆𝐼−𝜔₁𝑈𝐼−𝜔₂𝐿𝑥. (13) Let 𝜂_𝑘=𝜏−𝜔_𝑘+𝜆𝜔_𝑘,𝑘=1,2; by calculation, 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥+(1−𝜆)𝜔₁𝜔₂𝑈𝐿𝑥=[𝜆−(1−𝜏)]𝑥; (14) that is, 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥=(𝜆−1)𝜔₁𝜔₂𝑈𝐿𝑥+[𝜆−(1−𝜏)]𝑥. (15)
(1) Since 𝐵≥0 is irreducible, by Lemma 5, 𝜌(𝐵)>0. If 𝜌(𝑆_{𝜏,𝜔1,𝜔2})≥1, by 0≤𝜔_𝑘<𝜏≤1,𝑘=1,2, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})≥1>1−𝜏. If 𝜌(𝑆_{𝜏,𝜔1,𝜔2})<1, then 𝜆−(1−𝜏)≥0 because the left side of (14) is nonnegative, Thus 𝜆≥1−𝜏. By (14), 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥≤[𝜆−(1−𝜏)]𝑥. (16) If 𝜆=1−𝜏, by (16), we have 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥≤0; that is, 𝜂₁^𝑛_𝑗=𝑖+1𝑏_𝑖𝑗𝑥_𝑗+𝜂₂^𝑖−1_𝑗=1𝑏_𝑖𝑗𝑥_𝑗≤0,∀𝑖. (17)
Since 𝜂_𝑘>0,𝑘=1,2,𝐵=(𝑏_𝑖𝑗)≥0 and 𝑥>0, we obtain that 𝐵=0. Thus, 𝜌(𝐵)=0. This contradicts 𝜌(𝐵)>0. So, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1−𝜏.
(2) For mutually exclusive relations, consider the following.
(i) If 0<𝜌(𝐵)<1, let 𝑀=𝐼−𝜔₁𝑈𝐼−𝜔₂𝐿,𝑁=(1−𝜏)𝐼+𝜏−𝜔₁𝑈+𝜏−𝜔₂𝐿+𝜔₁𝜔₂𝑈𝐿, (18) and then 𝑆_{𝜏,𝜔1,𝜔2}=𝑀⁻¹𝑁. (19)
Since 𝑀⁻¹=(𝐼−𝜔₂𝐿)⁻¹(𝐼−𝜔₁𝑈)⁻¹≥0 and 𝑁≥0, 𝑇=𝑀−𝑁 is a regular splitting: 𝑇=𝑀−𝑁=𝜏𝐼−𝜏(𝑈+𝐿)=𝜏(𝐼−(𝑈+𝐿))=𝜏(𝐼−𝐵). (20)
By 𝐵≥0,0<𝜌(𝐵)<1, and 0<𝜏≤1, we know that 𝑇⁻¹=(1/𝜏)(𝐼−𝐵)⁻¹≥0. By Lemma 4, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})=𝜌(𝑀⁻¹𝑁)<1. Combine this with the result in (1), we have 1−𝜏<𝜌(𝑆_{𝜏,𝜔1,𝜔2})<1.
If 1−𝜏<𝜆=𝜌(𝑆_{𝜏,𝜔1,𝜔2})<1, by (14), we have 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥≤[𝜆−(1−𝜏)]𝑥. (21) Since 𝜂_𝑘>0,𝑘=1,2,𝑏_𝑖𝑖=0,∀𝑖, min_𝑘=1,2𝜂_𝑘^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗≤𝜂₁^𝑛_𝑗=𝑖+1𝑏_𝑖𝑗𝑥_𝑗+𝜂₂^𝑖−1_𝑗=1𝑏_𝑖𝑗𝑥_𝑗≤[𝜆−(1−𝜏)]𝑥_𝑖,∀𝑖; (22) that is, ∑^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗𝑥_𝑖≤𝜆−(1−𝜏)min_𝑘=1,2𝜂_𝑘,∀𝑖. (23)
By 𝜆<1 and 1−𝜔_𝑘>0(𝑘=1,2), there is 𝜆1−𝜔_𝑘<1−𝜔_𝑘,𝜆−1<−𝜔_𝑘+𝜆𝜔_𝑘,0<𝜆−(1−𝜏)=𝜆−1+𝜏<𝜂_𝑘. (24) Thus, ()0<𝜆−1−𝜏𝜂_𝑘<1,𝑘=1,2. (25)
Combining (23) with (29), we have ∑^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗𝑥_𝑖<1,∀𝑖. (26) By Lemma 6, we obtain that 0<𝜌(𝐵)<1.
(ii) If 𝜆=𝜌(𝑆_{𝜏,𝜔1,𝜔2})=1, by (14), we have 𝜏(𝑈+𝐿)𝑥=[1−(1−𝜏)]𝑥=𝜏𝑥; (27) namely, 𝐵𝑥=𝑥. Since 𝑥>0, we have ∑^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗𝑥_𝑖=1,∀𝑖. (28) By Lemma 6, we obtain that 𝜌(𝐵)=1.
(iii) If 𝜆=𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1, by (15), we have 𝜂₁𝑈𝑥+𝜂₂𝐿𝑥≥[𝜆−(1−𝜏)]𝑥. (29) Since 𝜂_𝑘>0,𝑘=1,2,𝑏_𝑖𝑖=0,∀𝑖, max_𝑘=1,2𝜂_𝑘^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗≥𝜂₁^𝑛_𝑗=𝑖+1𝑏_𝑖𝑗𝑥_𝑗+𝜂₂^𝑖−1_𝑗=1𝑏_𝑖𝑗𝑥_𝑗≥[𝜆−(1−𝜏)]𝑥_𝑖,∀𝑖; (30) that is, ∑^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗𝑥_𝑖≥𝜆−(1−𝜏)max_𝑘=1,2𝜂_𝑘,∀𝑖. (31)
By 𝜆>1 and 1−𝜔_𝑘>0(𝑘=1,2), there is 𝜆1−𝜔_𝑘>1−𝜔_𝑘,𝜆−1>−𝜔_𝑘+𝜆𝜔_𝑘,𝜆−(1−𝜏)=𝜆−1+𝜏>𝜂_𝑘>0. (32) Thus, 𝜆−(1−𝜏)𝜂_𝑘>1,𝑘=1,2. (33)
Combining (31) with (33), we have ∑^𝑛_𝑗=1𝑏_𝑖𝑗𝑥_𝑗𝑥_𝑖>1,∀𝑖. (34) By Lemma 6, we obtain that 𝜌(𝐵)>1.
If 𝜌(𝐵)=1 and 𝜌(𝑆_{𝜏,𝜔1,𝜔2})≠1, by (1), we obtain that 1−𝜏<𝜆=𝜌(𝑆_{𝜏,𝜔1,𝜔2})<1 or 𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1. Thus, by (i) and (iii), we know that 0<𝜌(𝐵)<1 or 𝜌(𝐵)>1. This contradicts 𝜌(𝐵)=1. So, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})=1.
If 𝜌(𝐵)>1 and 𝜌(𝑆_{𝜏,𝜔1,𝜔2})≤1, by (i) and (ii), we have 𝜌(𝐵)≤1. This contradicts 𝜌(𝐵)>1. So, 𝜌(𝑆_{𝜏,𝜔1,𝜔2})>1.

With special values of 𝜔₁, 𝜔₂, and 𝜏, we have the following corollaries.

Corollary 8. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_𝜔,0,0 the backward JOR iterative matrix. Then, for 0≤𝜔≤1, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_𝜔,0,0)>1−𝜔. (2)One and only one of the following mutually exclusive relations is valid. (i)0<𝜌(𝐵)<1⇔1−𝜔<𝜌(𝑆_𝜔,0,0)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_𝜔,0,0)=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_𝜔,0,0)>1.

Thus, The Jacobi iterative method and the backward JOR iterative method are either both convergent or both divergent.

Corollary 9. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_1,1,0 the backward Gauss-Seidel iterative matrix. Then, we have the following.
 (1)𝜌(𝐵)>0, 𝜌(𝑆_1,1,0)>0. (2)One and only one of the following mutually exclusive relations is valid.(i)0<𝜌(𝐵)<1⇔0<𝜌(𝑆_1,1,0)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_1,1,0)=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_1,1,0)>1.

Thus, The Jacobi iterative method and the backward Gauss-Seidel iterative method are either both convergent or both divergent.

Corollary 10. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_𝜔,𝜔,0 the backward SOR iterative matrix. Then, for 0≤𝜔≤1, we have the following.
 (1)𝜌(𝐵)>0, 𝜌(𝑆_𝜔,,𝜔,0)>1−𝜔. (2)One and only one of the following mutually exclusive relations is valid.  (i)0<𝜌(𝐵)<1⇔1−𝜔<𝜌(𝑆_𝜔,,𝜔,0)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_𝜔,𝜔,0)=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_𝜔,𝜔,0)>1.

Thus, The Jacobi iterative method and the backward SOR iterative method are either both convergent or both divergent.

Corollary 11. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_𝛼,𝜔,0 the backward AOR iterative matrix. Then, for 0≤𝜔<𝛼≤1, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_𝛼,𝜔,0)>1−𝛼. (2)One and only one of the following mutually exclusive relations is valid.(i)0<𝜌(𝐵)<1⇔1−𝛼<𝜌(𝑆_𝛼,𝜔,0)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_𝛼,𝜔,0)=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_𝛼,𝜔,0)>1.

Thus, The Jacobi iterative method and the backward AOR iterative method are either both convergent or both divergent.

Corollary 12. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_{𝜔(2−𝜔),𝜔,𝜔} the backward SSOR iterative matrix. Then, for 0≤𝜔≤1, we have the following.
 (1)𝜌(𝐵)>0, 𝜌(𝑆_{𝜔(2−𝜔),𝜔,𝜔})>(1−𝜔)². (2)One and only one of the following mutually exclusive relations is valid.  (i)0<𝜌(𝐵)<1⇔(1−𝜔)²<𝜌(𝑆_{𝜔(2−𝜔),𝜔,𝜔})<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_{𝜔(2−𝜔),𝜔,𝜔})=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_{𝜔(2−𝜔),𝜔,𝜔})>1.

Thus, The Jacobi iterative method and the backward SSOR iterative method are either both convergent or both divergent.

Corollary 13. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_{𝜔,𝜔,𝜔} the backward EMA iterative matrix. Then, for 0≤𝜔≤1, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_{𝜔,𝜔,𝜔})>(1−𝜔)². (2)One and only one of the following mutually exclusive relations is valid.  (i)0<𝜌(𝐵)<1⇔(1−𝜔)²<𝜌(𝑆_{𝜔,𝜔,𝜔})<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_{𝜔,𝜔,𝜔})=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_{𝜔,𝜔,𝜔})>1.

Thus, The Jacobi iterative method and the backward EMA iterative method are either both convergent or both divergent.

Corollary 14. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_{𝛼,𝜔,𝜔} the backward PSD iterative matrix. Then, for 0≤𝜔<𝛼≤1, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_{𝛼,𝜔,𝜔})>1−𝛼. (2)One and only one of the following mutually exclusive relations is valid. (i)0<𝜌(𝐵)<1⇔1−𝛼<𝜌(𝑆_𝛼,𝜔,0)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_{𝛼,𝜔,𝜔})=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_{𝛼,𝜔,𝜔})>1.

Thus, The Jacobi iterative method and the backward PSD iterative method are either both convergent or both divergent.

Corollary 15. Let the coefficient matrix 𝐴 of (1) be irreducible, 𝐵=𝑈+𝐿≥0 the Jacobi matrix, and 𝑆_1,𝜔,𝜔 the backward PJ iterative matrix. Then, for 0≤𝜔≤1, we have the following. (1)𝜌(𝐵)>0, 𝜌(𝑆_1,𝜔,𝜔)>0. (2)One and only one of the following mutually exclusive relations is valid.    (i)0<𝜌(𝐵)<1⇔0<𝜌(𝑆_1,𝜔,𝜔)<1.(ii)𝜌(𝐵)=1⇔𝜌(𝑆_1,𝜔,𝜔)=1.(iii)𝜌(𝐵)>1⇔𝜌(𝑆_1,𝜔,𝜔)>1.

Thus, The Jacobi iterative method and the backward PJ iterative method are either both convergent or both divergent.

Remark 16. The convergence results between the backward MPSD and Jacobi iterative matrix are proposed, and The convergence results between some special cases of backward MPSD (such as backward JOR, backward G-S, backward EMA, and backward PSD) and Jacobi iterative matrix are obtained. These results involve some special iterative methods which are proposed in the references.

4. Numerical Examples

In this section, we show five examples. The first three examples are used to show the convergence of the proposed iterative methods. Example 4 is used to show the divergence of the proposed iterative methods. Example 5 shows that the backward iterative methods are better than the origin methods when the upper triangular part dominates the lower triangular part. In the following figures, horizontal axis denotes the numbers of iterations and vertical axis denotes the errors of iterations.

Example 1. Let the coefficient matrix 𝐴 and the vector 𝑏 of (1) be 1𝐴=10−4−14101−4−14−14−14−1104−141111.01,𝑏= (35)
The Jacobi iterative matrix is 1𝐵=004141004141414100414.00 (36)
By caculation, we obtain 0<𝜌(𝐵)=1/2<1.(1)Let 𝜏=𝛼=1/2, 𝜔₁=𝜔₂=𝜔=1/4. We obtain the backward PSD iterative matrix 𝑆_{𝛼,𝜔,𝜔}=𝑆_1/2,1/4,1/4, and 1>𝜌𝑆_{𝛼,𝜔,𝜔}=𝜌𝑆_1/2,1/4,1/4=967>113492=1−𝛼. (37)(2)Let 𝜏=1, 𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward PJ iterative matrix 𝑆_1,𝜔,𝜔=𝑆_1,1/2,1/2, and 1>𝜌𝑆_1,𝜔,𝜔=𝜌𝑆_1,1/2,1/2=250693>0=1−𝜏. (38)(3)Let 𝜏=𝜔=1/2, 𝜔₁=𝜔₂=0. We obtain the backward JOR iterative matrix 𝑆_𝜔,0,0=𝑆_1/2,0,0, and 1>𝜌𝑆_𝜔,0,0=𝜌𝑆_1/2,0,0=34>12=1−𝜔. (39)(4)Let 𝜏=𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward EMA iterative matrix 𝑆_{𝜔,𝜔,𝜔}=𝑆_1/2,1/2,1/2, and 1>𝜌𝑆_{𝜔,𝜔,𝜔}=𝜌𝑆_1/2,1/2,1/2=943>113864=(1−𝜔)². (40) With these iterative methods and the presented algorithm, the solution is 𝐱=(2,2,2,2)^𝑇.

Example 2. In order to obtain the numerical solution of the laplace equation 𝜕²𝑢𝑥,𝑦𝜕𝑥²+𝜕²𝑢𝑥,𝑦𝜕𝑦²=𝑢_𝑥𝑥𝑥,𝑦+𝑢_𝑦𝑦𝑥,𝑦=0, (41) under a uniform square mesh of five-point difference approximations, and the interior mesh points as shown in Figure 1 [28], we can obtain the linear system (1), where the matrix 𝐴 and the vector 𝑏 of (1) are1𝐴=1000000−410−4−1410100000−41000001000−4−14−14−1401000100−4−14100000001000−4−1401000001000−4−14100−4−14−100100004−14−14−141000100000−410−4−1000100410−410−4−14−100010410000−411111111111.00001,𝑏= (42)
The Jacobi iterative matrix is 1𝐵=00000004014141000000041000000000414141401000000414100000000000414010000000004−141004141001000041414141000000000401410001004014014141000004100004.00000 (43) By caculation, we obtain 0<𝜌(𝐵)=1159/1601<1.(1)Let 𝜏=𝛼=1/2, 𝜔₁=𝜔₂=𝜔=1/4. We obtain the backward PSD iterative matrix 𝑆_{𝛼,𝜔,𝜔}=𝑆_1/2,1/4,1/4, and 1>𝜌𝑆_{𝛼,𝜔,𝜔}=𝜌𝑆_1/2,1/4,1/4=1061>112712=1−𝛼. (44)(2)Let 𝜏=1, 𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward PJ iterative matrix 𝑆_1,𝜔,𝜔=𝑆_1,1/2,1/2, and 1>𝜌𝑆_1,𝜔,𝜔=𝜌𝑆_1,1/2,1/2=13192168>0=1−𝜏. (45)(3)Let 𝜏=𝜔=1/2, 𝜔₁=𝜔₂=0. We obtain the backward JOR iterative matrix 𝑆_𝜔,0,0=𝑆_1/2,0,0, and 1>𝜌𝑆_𝜔,0,0=𝜌𝑆_1/2,0,0=1380>116012=1−𝜔. (46)(4)Let 𝜏=𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward EMA iterative matrix 𝑆_{𝜔,𝜔,𝜔}=𝑆_1/2,1/2,1/2, and 1>𝜌𝑆_{𝜔,𝜔,𝜔}=𝜌𝑆_1/2,1/2,1/2=3602>144794=(1−𝜔)². (47)
From Figures 2, 3, 4, and 5, the errors of Jacobi iteration are denoted by blue circles and that of MPSD iteration is denoted by red stars. By the figures above, We know that Jacobi iteration is better than backward PSD, JOR, and EMA iteration and is worse than PJ iteration under the values of 𝜔₁, 𝜔₂, and 𝜏 in this example.

Figure 1 

Uniform square mesh of five-point difference.

Figure 2 

The errors of PSD and Jacobi iteration.

Figure 3 

The errors of PJ and Jacobi iteration.

Figure 4 

The errors of JOR and Jacobi iteration.

Figure 5 

The errors of EMA and Jacobi iteration.

Example 3. Let the coefficient matrix 𝐴 of (1) be 𝐴=,1𝑞𝑟𝑠𝑞⋯𝑠1𝑞𝑟⋱𝑞𝑞𝑠⋱⋱⋱𝑠𝑟⋱⋱1𝑞𝑟𝑠⋱𝑞𝑠1𝑞⋯𝑠𝑟𝑞𝑠1 (48) where 𝑞=−𝑝/𝑛, 𝑟=−𝑝/(𝑛+1), and 𝑠=−𝑝/(𝑛+2) [28]. Here, we let 𝑛=100 and 𝑝=1. By caculation, we obtain 0<𝜌(𝐵)=199/203<1.(1)Let 𝜏=𝛼=1/2, 𝜔₁=𝜔₂=𝜔=1/4. We obtain the backward PSD iterative matrix 𝑆_{𝛼,𝜔,𝜔}=𝑆_1/2,1/4,1/4, and 1>𝜌𝑆_{𝛼,𝜔,𝜔}=𝜌𝑆_1/2,1/4,1/4=233>12362=1−𝛼. (49)(2)Let 𝜏=1, 𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward PJ iterative matrix 𝑆_1,𝜔,𝜔=𝑆_1,1/2,1/2, and 1>𝜌𝑆_1,𝜔,𝜔=𝜌𝑆_1,1/2,1/2=724749>0=1−𝜏. (50)(3)Let 𝜏=𝜔=1/2, 𝜔₁=𝜔₂=0. We obtain the backward JOR iterative matrix 𝑆_𝜔,0,0=𝑆_1/2,0,0, and 1>𝜌𝑆_𝜔,0,0=𝜌𝑆_1/2,0,0=199>12032=1−𝜔. (51)(4)Let 𝜏=𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward EMA iterative matrix 𝑆_{𝜔,𝜔,𝜔}=𝑆_1/2,1/2,1/2, and 1>𝜌𝑆_{𝜔,𝜔,𝜔}=𝜌𝑆_1/2,1/2,1/2=766>17794=(1−𝜔)². (52)

Example 4. Let the coefficient matrix 𝐴 and the vector 𝑏 of (1) be 212𝐴=.1−2−2−11−1−2−21,𝑏= (53) The Jacobi iterative matrix is 𝐵=.022101220 (54) By caculation, we obtain 𝜌(𝐵)=987/305>1.(1)Let 𝜏=𝛼=1/2, 𝜔₁=𝜔₂=𝜔=1/4. We obtain the backward PSD iterative matrix 𝑆_{𝛼,𝜔,𝜔}=𝑆_1/2,1/4,1/4, and 𝜌𝑆_{𝛼,𝜔,𝜔}=𝜌𝑆_1/2,1/4,1/4=36631052>1. (55)(2)Let 𝜏=1, 𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward PJ iterative matrix 𝑆_1,𝜔,𝜔=𝑆_1,1/2,1/2, and 𝜌𝑆_1,𝜔,𝜔=𝜌𝑆_1,1/2,1/2=2543247>1. (56)(3)Let 𝜏=𝜔=1/2, 𝜔₁=𝜔₂=0. We obtain the backward JOR iterative matrix 𝑆_𝜔,0,0=𝑆_1/2,0,0, and 𝜌𝑆_𝜔,0,0=𝜌𝑆_1/2,0,0=646305>1. (57)(4)Let 𝜏=𝜔₁=𝜔₂=𝜔=1/2. We obtain the backward EMA iterative matrix 𝑆_{𝜔,𝜔,𝜔}=𝑆_1/2,1/2,1/2, and 𝜌𝑆_{𝜔,𝜔,𝜔}=𝜌𝑆_1/2,1/2,1/2=1530271>1. (58) It shows that the backward MPSD iteration is invalid for this example.

Example 5. Let the coefficient matrix 𝐴 of (1) be𝐴=.10−2−2−2−2−2−2−2−2−2−110−2−2−2−2−2−2−2−2−1−111−2−2−2−2−2−2−2−1−1−112−2−2−2−2−2−2−1−1−1−113−2−2−2−2−2−1−1−1−1−114−2−2−2−2−1−1−1−1−1−115−2−2−2−1−1−1−1−1−1−116−2−2−1−1−1−1−1−1−1−117−2−1−1−1−1−1−1−1−1−118 (59)
We can see the analogous matrix 𝐴 in [29]. By caculation, we obtain 𝜌(𝐵)=811/822<1.

Let 𝜏=1, 𝜔₁=1,𝜔₂=0. We obtain the backward Gauss-Seidel iterative matrix 𝑆_{1,𝜔1,𝜔2}=𝑆_1,1,0, and 𝜌𝑆_{1,𝜔1,𝜔2}=𝜌𝑆_1,1,0=766.789 (60) The Gauss-Seidel iterative matrix 𝑆_{1,𝜔1,𝜔2}=𝑆_1,1,0=(𝐼−𝐿)⁻¹𝑈, and 𝜌𝑆_{1,𝜔1,𝜔2}=𝜌𝑆_1,1,0=721.739 (61) Thus, 𝜌𝑆_1,1,0<𝜌𝑆_1,1,0. (62)

From Figure 6, the red stars denote the error of backward Guass-Seidel iteration and the blue circles denote that of Guass-Seidel. So, the backward iterative methods are better than the original methods under the assumption that the upper triangular part dominates the lower triangular part.

Figure 6 

The errors of backward Guass-Seidel and Guass-Seidel iteration.

5. Conclusions

The Jacobi iteration is the basic iteration for linear systems and easier to the analysis of the convergence than other iterations. In the paper, we proposed the backward MPSD iteration and obtained the convergence result between backward MPSD iteration (including iterations such as backward JOR, backward G-S, backward EMA, and backward PSD) and Jacobi iteration. We pointed out that the backward MPSD iteration and the Jacobi iteration are either both convergent or both divergent under the assumptions in Theorem 7. So, we can ascertain the convergence or divergence of backward MPSD iteration by Jacobi iteration. In some case, the backward iteration is better than the original one.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The work is partially supported by National Nature Science Foundation of China (1117105, 11101071, and 61001200) and the Fundamental Research Funds for the centeral Universities (ZYGX2009J103).