Different Versions of ILU and IUL Factorizations Obtained from Forward and Backward Factored Approximate Inverse Processes—Part I

Abstract

We present an incomplete UL (IUL) decomposition of matrix 𝐴 which is extracted as a by-product of BFAPINV (backward factored approximate inverse) process. We term this IUL factorization as IULBF. We have used ILUFF [3] and IULBF as left preconditioner for linear systems. Different versions of ILUFF and IULBF preconditioners are computed by using different dropping techniques. In this paper, we compare quality of different versions of ILUFF and IULBF preconditioners.

1. Introduction

Consider the linear system of equations 𝐴𝑋=𝑏, (1.1) where the coefficient matrix 𝐴∈𝑅^𝑛×𝑛 is nonsymmetric, nonsingular, large, sparse, and 𝑋,𝑏∈𝑅^𝑛. Suppose 𝑀≈𝐴. Linear system 𝑀⁻¹𝐴𝑋=𝑀⁻¹𝑏, (1.2) is termed left preconditioned system of system (1.1) and matrix 𝑀 is called left preconditioner matrix [1]. System (1.2) is solved by Krylov subspace methods [1].

Suppose that matrix 𝐴 is nonsymmetric. Also, suppose that 𝑊=[𝑤^𝑇₁,…,𝑤^𝑇_𝑛]^𝑇 and 𝑍=[𝑧₁,…,𝑧_𝑛] are unit lower and upper triangular matrices, respectively, and 𝐷=diag(𝑑₁,…,𝑑_𝑛) is a diagonal matrix. FFAPINV (forward factored approximate inverse) Algorithm [2], computes matrices 𝑊, 𝑍, and 𝐷 such that relation 𝑊𝐴𝑍≈𝐷, (1.3) holds. It is possible to obtain an incomplete 𝐿𝑈 (ILU) decomposition of matrix 𝐴, as a by-product of FFAPINV process, such that 𝐿 is an unit lower triangular and 𝑈 is an upper triangular matrix and 𝐴≈𝑀=𝐿𝑈. (1.4) Matrix 𝑀 in (1.4) is called ILUFF preconditioner (ILU factorization obtained from forward factored approximate inverse process) [3]. The approximate inverse factors 𝑊, 𝑍 and 𝐷 in (1.3) and 𝐿, 𝑈 matrices in (1.4) satisfy the two following relations: 𝐿≈𝑊⁻¹,𝑈≈𝐷𝑍⁻¹. (1.5)

In Algorithms 1 and 2, 𝐴_∶,𝑗 and 𝐴_𝑗,∶ refer to 𝑗th column and 𝑗th row of matrix A, respectively.

(1) 𝑤1=𝑒𝑇1,𝑧1=𝑒1,𝑑1=𝑎11. (2) for 𝑗=2 to 𝑛  do (3)  𝑤𝑗=𝑒𝑇𝑗,𝑧𝑗=𝑒𝑗. (4)  for  𝑖=1 to 𝑗−1 do (5)   𝐿𝑗𝑖=𝐴𝑗,∶𝑧𝑖𝑑𝑖,𝑈𝑖𝑗=𝑤𝑖𝐴∶,𝑗𝑑𝑖 (6)   apply a dropping rule to 𝐿𝑗𝑖 and to 𝑈𝑖𝑗 (7)   𝑧𝑗=𝑧𝑗−𝑤𝑖𝐴∶,𝑗𝑑𝑖𝑧𝑖,  𝑤𝑗=𝑤𝑗−𝐴𝑗,∶𝑧𝑖𝑑𝑖𝑤𝑖 (8)   for all 𝑙≤𝑖 apply a dropping rule to 𝑧𝑙𝑗 and to 𝑤𝑗𝑙      (first format of dropping for 𝑊 and 𝑍) (9)   end for (10) for all 𝑙≤𝑗 apply a dropping rule to 𝑧𝑙𝑗 and to 𝑤𝑗𝑙       (second format of dropping for 𝑊 and 𝑍) (11)  𝑑𝑗=𝑤𝑗𝐴∶,𝑗 (if 𝐴 is not positive definite) (12)  𝑑𝑗=𝑤𝑗𝐴𝑤𝑇𝑗 (if 𝐴 is positive definite) (13) end for (14) Return 𝐿=(𝐿𝑖𝑗) and 𝑈=(𝑑𝑖𝑈𝑖𝑗)

Algorithm 1 

ILUFF algorithm.

(1) 𝑤𝑛=𝑒𝑇𝑛,𝑧𝑛=𝑒𝑛,𝑑𝑛=𝑎𝑛𝑛. (2) for  𝑗=𝑛−1 to 1 do (3)  𝑤𝑗=𝑒𝑇𝑗,𝑧𝑗=𝑒𝑗. (4)  for 𝑖=𝑗+1 to 𝑛  do (5)   𝑈𝑗𝑖=𝐴𝑗,∶𝑧𝑖𝑑𝑖,𝐿𝑖𝑗=𝑤𝑖𝐴∶,𝑗𝑑𝑖 (6)   apply a dropping rule to 𝑈𝑗𝑖 and to 𝐿𝑖𝑗 (7)   𝑧𝑗=𝑧𝑗−𝑤𝑖𝐴∶,𝑗𝑑𝑖𝑧𝑖,𝑤𝑗=𝑤𝑗−𝐴𝑗,∶𝑧𝑖𝑑𝑖𝑤𝑖 (8)   for all 𝑙≥𝑖 apply a dropping rule to 𝑧𝑙𝑗 and to 𝑤𝑗𝑙     (first format of dropping for 𝑊 and 𝑍) (9)   end for (10) for all 𝑙≥𝑗 apply a dropping rule to 𝑧𝑙𝑗 and to 𝑤𝑗𝑙      (second format of dropping for 𝑊 and 𝑍) (11)  𝑑𝑗=𝑤𝑗𝐴∶,𝑗 (if 𝐴 is not positive definite) (12)  𝑑𝑗=𝑤𝑗𝐴𝑤𝑇𝑗 (if 𝐴 is positive definite) (13) end for (14) Return 𝐿=(𝑑𝑗𝐿𝑖𝑗) and 𝑈=(𝑈𝑖𝑗)

Algorithm 2 

IULBF algorithm.

In Section 2 of this paper, we present different dropping strategies for 𝑊, 𝑍 and 𝐿, 𝑈 factors of ILUFF preconditioner. In Section 3, we first introduce the IULBF preconditioner and then, we present different dropping strategies for this preconditioner. In Section 4, we present numerical results.

2. Different Versions of ILUFF Preconditioner

Algorithm 1,which has been presented in the next page, computes the ILUFF preconditioner.

Suppose that 𝜀_𝑍 and 𝜀_𝑊 are the drop tolerance parameters for 𝑍 and 𝑊 matrices, respectively. We have used two strategies to drop entries of 𝑧_𝑗 and 𝑤_𝑗 vectors in ILUFF algorithm.

(i) First Dropping Strategy
In this strategy, only line 8 of Algorithm 1 will be run and line 10 will not. In this case, entries 𝑧_𝑙𝑗 and 𝑤_𝑗𝑙, for 𝑙≤𝑖<𝑗 are dropped when 𝑧_𝑙𝑗≤𝜀_𝑍,𝑤_𝑗𝑙≤𝜀_𝑊. (2.1)

(ii) Second Dropping Strategy
In this strategy, only line 10 of Algorithm 1 will be run and line 8 will not. In this case, the whole vectors 𝑧_𝑗 and 𝑤_𝑗 are computed as 𝑧_𝑗=𝑒_𝑗−^𝑗−1_𝑖=1𝑤_𝑖𝐴_∶,𝑗𝑑_𝑖𝑧_𝑖,𝑤_𝑗=𝑒^𝑇_𝑗−^𝑗−1_𝑖=1𝐴_𝑗,∶𝑧_𝑖𝑑_𝑖𝑤_𝑖, (2.2) and then, entries 𝑤_𝑗𝑙 and 𝑧_𝑙𝑗, for 𝑙≤𝑗, are dropped when criterions (2.1) are satisfied.

We have used two strategies to drop entries of 𝐿 and 𝑈 matrices in ILUFF algorithm.

(i) Inverse-Based Dropping Strategy
Let 𝜀_𝐿,𝑊 be the same drop tolerance parameter for 𝐿 and 𝑊 matrices and 𝜀_𝑈,𝑍 be the same drop tolerance parameter for 𝑈 and 𝑍 matrices. Consider 𝜀_𝐿,𝑊 as 𝜀_𝑊 and 𝜀_𝑈,𝑍 as 𝜀_𝑍. We drop entries 𝑧_𝑙𝑗 and 𝑤_𝑗𝑙, for 𝑙≤𝑖<𝑗, when criterions (2.1) hold. Then, in line 6 of Algorithm 1, entries 𝐿_𝑗𝑖 and 𝑈_𝑖𝑗, for 𝑖<𝑗, are dropped when 𝐿_𝑗𝑖𝑊_𝑖,∶1≤𝜀_𝐿,𝑊,𝑈_𝑖𝑗𝑍_∶,𝑖∞≤𝜀_𝑈,𝑍. (2.3)

(ii) Simple Dropping Strategy
Let 𝜀_𝐿 and 𝜀_𝑈 be the drop tolerance parameters for 𝐿 and 𝑈 matrices. In line 6 of Algorithm 1, entries 𝐿_𝑗𝑖 and 𝑈_𝑖𝑗, for 𝑖<𝑗, are dropped when 𝐿_𝑗𝑖≤𝜀_𝐿,𝑈_𝑖𝑗≤𝜀_𝑈. (2.4) Different versions of ILUFF preconditioners are computed by using different dropping strategies in Algorithm 1.

(i) ILUFF1
In Algorithm 1, first dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and simple dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(ii) ILUFF2
In Algorithm 1, first dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and inverse-based dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(iii) ILUFF3
In Algorithm 1, second dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and simple dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(iv) ILUFF4
In Algorithm 1, second dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and inverse-based dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

3. IULBF Preconditioner and Its Different Versions

Suppose that 𝑊=[𝑤^𝑇₁,…,𝑤^𝑇_𝑛]^𝑇 and 𝑍=[𝑧₁,…,𝑧_𝑛] are unit upper and lower triangular matrices, respectively, and 𝐷=diag(𝑑₁,…,𝑑_𝑛) is a diagonal matrix. BFAPINV algorithm [2, 4] computes matrices 𝑊, 𝑍, and 𝐷 such that relation (1.3) holds. We obtain an IUL decomposition of matrix 𝐴, as a by-product of BFAPINV process, such that 𝐿 is a lower triangular and 𝑈 is an unit upper triangular matrix and 𝐴≈𝑀=𝑈𝐿. (3.1)

Matrix 𝑀 in relation (3.1) is called IULBF preconditioner (IUL factorization obtained from backward factored approximate inverse process). Algorithm 2 computes the IULBF preconditioner. The approximate inverse factors 𝑊, 𝑍, and 𝐷 in (1.3) and 𝐿, 𝑈 matrices in (3.1) satisfy the two following relations: 𝑈≈𝑊⁻¹,𝐿≈𝐷𝑍⁻¹. (3.2)

Suppose that 𝜀_𝑍 and 𝜀_𝑊 are the drop tolerance parameters for 𝑍 and 𝑊 matrices, respectively. We have used two strategies to drop entries of 𝑧_𝑗 and 𝑤_𝑗 vectors in IULBF algorithm.

(i) First Dropping Strategy
In this strategy, only line 8 of Algorithm 2 will be run and line 10 will not. In this case, entries 𝑧_𝑙𝑗 and 𝑤_𝑗𝑙, for 𝑗<𝑖≤𝑙 are dropped when criterions 𝑧_𝑙𝑗≤𝜀_𝑍,𝑤_𝑗𝑙≤𝜀_𝑊, (3.3) hold.

(ii) Second Dropping Strategy
In this strategy, only line 10 of Algorithm 2 will be run and line 8 will not. In this case, the whole vectors 𝑧_𝑗 and 𝑤_𝑗 are computed as 𝑧_𝑗=𝑒_𝑗−^𝑛_𝑖=𝑗+1𝑤_𝑖𝐴_∶,𝑗𝑑_𝑖𝑧_𝑖,𝑤_𝑗=𝑒^𝑇_𝑗−^𝑛_𝑖=𝑗+1𝐴_𝑗,∶𝑧_𝑖𝑑_𝑖𝑤_𝑖, (3.4) and then, entries 𝑤_𝑗𝑙 and 𝑧_𝑙𝑗, for 𝑙≥𝑗, are dropped when criterions (3.3) are satisfied.

We have used two strategies to drop entries of 𝐿 and 𝑈 matrices in IULBF algorithm.

(i) Inverse-Based Dropping Strategy
Let 𝜀_𝑈,𝑊 be the same drop tolerance parameter for 𝑈 and 𝑊 matrices and 𝜀_𝐿,𝑍 be the same drop tolerance parameter for 𝐿 and 𝑍 matrices. Consider 𝜀_𝑈,𝑊 as 𝜀_𝑊 and 𝜀_𝐿,𝑍 as 𝜀_𝑍. We drop entries 𝑧_𝑙𝑗 and 𝑤_𝑗𝑙, for 𝑗<𝑖≤𝑙, when criterions (3.3) hold. Then, in line 6 of Algorithm 2, entries 𝐿_𝑖𝑗 and 𝑈_𝑗𝑖, for 𝑖>𝑗, are dropped when𝐿_𝑖𝑗𝑍_∶,𝑖∞≤𝜀_𝐿,𝑍,𝑈_𝑗𝑖𝑊_𝑖,∶1≤𝜀_𝑈,𝑊. (3.5)

(ii) Simple Dropping Strategy
Let 𝜀_𝐿 and 𝜀_𝑈 be the drop tolerance parameters for 𝐿 and 𝑈 matrices. In line 6 of Algorithm 2, entries 𝐿_𝑖𝑗 and 𝑈_𝑗𝑖, for 𝑖>𝑗, are dropped when 𝐿_𝑖𝑗≤𝜀_𝐿,𝑈_𝑗𝑖≤𝜀_𝑈. (3.6) Different versions of IULBF preconditioner are computed by using different dropping strategies in Algorithm 2.

(i) IULBF1
In Algorithm 2, first dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and simple dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(ii) IULBF2
In Algorithm 2, first dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and inverse-based dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(iii) IULBF3
In Algorithm 2, second dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and simple dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

(iv) IULBF4
In Algorithm 2, second dropping strategy is used to drop entries of 𝑊 and 𝑍 matrices and inverse-based dropping strategy is used to drop entries of 𝐿 and 𝑈 matrices.

4. Numerical Results

In this section, we report results of left preconditioned GMRES(16) method [1]. Preconditioners are ILUFF1, ILUFF2, ILUFF3, ILUFF4, IULBF1, IULBF2, IULBF3, and IULBF4. All coefficient matrices are nonsymmetric and from University of Florida Sparse Matrix Collection [5]. Vector 𝑏 is 𝐴𝑒 in which 𝑒=[1,…,1]^𝑇. We have written codes of ILUFF1, ILUFF2, ILUFF3, ILUFF4, IULBF1, IULBF2, IULBF3, IULBF4, and GMRES(16) in MATLAB, and we have run all the experiments on a machine with 1GB of RAM memory. In all the experiments, if the pivot element 𝑑_𝑗 (lines 11 and 12 of ILUFF and IULBF algorithms) is less than the machine precision, then we replace it by 10⁻⁴. Density of preconditioners is defined as (𝐿)(𝑈)density=𝑛𝑛𝑧+𝑛𝑛𝑧𝑛𝑛𝑧(𝐴), (4.1)

in which 𝑛𝑛𝑧(𝐿), 𝑛𝑛𝑧(𝑈), and 𝑛𝑛𝑧(𝐴) refer to the number of nonzero entries of 𝐿,𝑈, and 𝐴 matrices, respectively. In all the experiments, we have selected 𝜀_𝐿,𝜀_𝑈, 𝜀_𝑊, 𝜀_𝑍, 𝜀_𝐿,𝑍, 𝜀_𝑈,𝑊, 𝜀_𝐿,𝑊, and 𝜀_𝑈,𝑍 equal to 0.1.

Table 1, reports results of GMRES(16) method without preconditioning. In this table, 𝑛 indicates the dimension of the matrix and 𝑃𝐷 column indicates whether or not the matrix is positive definite. Yes (no) in this column means that the matrix is (is not) positive definite. 𝐼time, indicates the iteration time of GMRES(16) without preconditioning and 𝑖𝑡, is the number of iterations of GMRES(16) method. 𝐼time is in seconds. In this table, + means that there is no convergence after 10,000 iterations. In all the experiments, the stopping criterion is 𝑟_𝑘2𝑟₀₂≤10⁻⁸, (4.2)

Table 1 

Information of GMRES(16) method without preconditioning and matrix properties.

in which 𝑟_𝑘 is the 𝑘th residual vector of the system and 𝑟₀ is the initial residual vector. In all the experiments, the initial guess is the zero vector.

In Table 2, the information of ILUFF1, ILUFF2, ILUFF3, and ILUFF4 preconditioners are presented and also in Table 3, the information of IULBF1, IULBF2, IULBF3, and IULBF4 preconditioners are presented. In Tables 2 and 3, 𝑃time is the preconditioning time and density is the density of preconditioner. 𝑃time is also in seconds.

Table 2 

properties of ILUFF1, ILUFF2, ILUFF3, and ILUFF4 preconditioners.

Table 3 

Properties of IULBF1, IULBF2, IULBF3, and IULBF4 preconditioners.

In Table 4, results of left preconditioned systems by using different versions of ILUFF preconditioner have been presented, and also in Table 5 results of left preconditioned systems by using different versions of IULBF preconditioner have been presented. In Tables 4 and 5, 𝑇time is the total time which is the sum of preconditioning time and iteration time, and 𝑖𝑡 is the number of iterations of left preconditioned GMRES(16). In these tables, + indicates that no convergence has been obtained in 5000 iterations.

Table 4 

Information of preconditioned GMRES(16) method by using different versions of ILUFF preconditioner.

Table 5 

Information of preconditioned GMRES(16) method by using different versions of IULBF preconditioner.

5. Conclusion

Results of Tables 1 and 4 show that ILUFF1, ILUFF2, ILUFF3, and ILUFF4 preconditioners are useful tools to decrease the number of iterations of GMRES(16) method and results of Tables 1 and 5 show that IULBF1, IULBF2, IULBF3, and IULBF4 preconditioners are also useful tools to decrease the number of iterations of GMRES(16) method.

Comparison of columns 2 and 6 of Table 4 indicates that sometimes ILUFF3 preconditioner decreases the number of iterations of GMRES(16) method a little bit more than ILUFF1 preconditioner and some other times it is vice versa. Comparison of columns 2 and 4 and columns 6 and 8 of this table, also shows that ILUFF2 preconditioner decreases the number of iterations of GMRES(16) method more than ILUFF1 preconditioner and ILUFF4 preconditioner decreases the number of iterations of GMRES(16) method more than ILUFF3 preconditioner.

Comparison of columns 2 and 6 of Table 5 indicates that IULBF3 preconditioner decreases the number of iterations of GMRES(16) method a little bit more than IULBF1 preconditioner. Comparison of columns 2 and 4 and columns 6 and 8 of this table, also shows that IULBF2 preconditioner decreases the number of iterations of GMRES(16) method more than IULBF1 preconditioner and IULBF4 preconditioner decreases the number of iterations of GMRES(16) method more than IULBF3 preconditioner.

Comparison of columns of Tables 4 and 5 indicate that (except for matrix epb0) different versions of IULBF preconditioner decrease the number of iterations of GMRES(16) method more than different versions of ILUFF preconditioner.