Abstract
In this paper, we mainly focus on the development and study of a new global GCRODR method that allows both the flexible preconditioning and the subspace recycling for sequences of shifted linear systems. The novel method presented here has two main advantages: firstly, it does not require the righthand sides to be related, and, secondly, it can also be compatible with the general preconditioning. Meanwhile, we apply the new algorithm to solve the general coupled matrix equations. Moreover, by performing an error analysis, we deduce that a much looser tolerance can be applied to save computation by limiting the flexible preconditioned work without sacrificing the closeness of the computed and the true residuals. Finally, numerical experiments demonstrate that the proposed method illustrated can be more competitive than some other global GMREStype methods.
1. Introduction
In the current study, we focus on the solution of sequences of shifted linear systems of the formwhere are large nonsingular matrices of order , the righthand sides are vectors of the length , and are called shifts. Such shifted linear systems (1) arise in many realistic applications, for example, electromagnetic scattering [1], QCD [2–5], dynamics of structures [6], complex network [7–9], and model reduction [10].
Note that problem (1) is reduced to the solution of a sequence of linear systems with the formwhen , where the coefficient matrix and the righthand sides change from one system to the next. One approach to more efficient solution of such systems is to exploit Krylov subspace recycling; that is, recycling a judiciously selected subspace of the search space typically reduces the number of iterations substantially. There are several algorithms that take advantage of recycling for subsequent systems as well as for restarts of generalized minimal residual (GMRES) type methods, such as the generalized conjugate residual method with inner orthogonalization (GCRO) [11], GCRO with deflated restarting (GCRODR) [12], GMRES with deflated restarting (GMRESDR) [13] (only for a single system), and GCRO with optimal truncation (GCROT) [14].
It is also noted that, for each , we substantially solve a family of shifted linear systems with the form
Krylov subspace solvers [15] are particularly attractive for the solutions of (3) because of its shiftinvariance property [16]. That is, for any shift , the th Krylov subspace satisfiesas long as . This property has led to many approaches for solving such linear systems (3) simultaneously by generating only one subspace. Some solvers are built upon the (Bi)Lanczos recurrences [17–21] and some others use the Arnoldilike iteration [16, 22–28]. However, the shiftinvariance property will be lost when using the general preconditioning, such as Jacobi preconditioning, SSOR preconditioning, ILU preconditioning, AMG preconditioning, and AINV preconditioning. Furthermore, when are unrelated, we cannot employ the shiftinvariance property. It means that all residuals are not simultaneously minimized whiling maintaining collinearity, for example, the GMREStype methods [16, 23, 29].
Therefore, we propose an alternative without exploiting the shiftinvariance. In fact, the solution of (3) is equivalent to solve a Sylvester equation [30],where and . Therefore, (1) can also be reformulated as a family of Sylvester equations with the slowly changing coefficient matrices
For simplicity of the presentation, a linear operator is defined as follows:
Applying the linear operator , we can rewrite (1) as
Moreover, we also assume that the linear operator is invertible; that is,
Obviously, the linear systems (8) can be seen as generalized version of (2). Therefore, based on the special structure of (1), we propose a new generalized global version of the GCRODR method that not only combines with any type of the preconditioning but also allows one to retain important spectral information generated during the solution of the th linear system and then to exploit such information to accelerate convergence of the iterations when solving the subsequent th system. Moreover, the proposed method can be combined with any form of preconditioning and does not require the righthand sides to maintain collinearity.
For simplicity, we denote by . In the following, we utilize the operator to present our flexible global GCRODR algorithms for solving (1).
The paper is organized as follows. A global version of GCRODR to solve (1) is derived in detail in Section 2. We also apply the flexible preconditioning operator at each iteration to improve the global GCRODR algorithm. By performing an error analysis, we deduce that a much looser tolerance can be applied to save computation by limiting the flexible preconditioned work without sacrificing the closeness of the computed and the true residuals. Meanwhile, we also propose a global version of the FGMRESDR method and then analyze the relationship between two flexible global methods. In Section 2.5, the flexible global GCRODR method is applied to solve the general coupled matrix equations. Section 3 numerically demonstrates the effectiveness of the novel method. Finally, some conclusions and remarks are summarized in Section 4.
2. Flexible Global GCRODR Method
GCRODR is a wellestablished Krylov subspace solver that paired with a harmonic Ritz vector recycling strategy [12]. GCRO [11] is a nested Krylov method, which uses GCR as the outer solver and the restarted GMRES as an inner method. In [12], Parks et al. propose GCRODR that exploits deflated restarting within the framework of GCRO for a family of slowly changing linear equations. Because of its superior numerical behavior, we introduce a global version of GCRODR to solve (1). Here, we not only employ the subspace recycling at each cycle but also apply the flexible preconditioning operator at each iteration. The proposed method is referred to as flexible global GCRODR (FGLGCRODR).
2.1. Flexible Global Arnoldi Process
We first present a global Arnoldi process used in the flexible setting to create an Forthonormal basis for the global Krylov subspace in Algorithm 1. We define as the preconditioning operator used at the th iteration, and the action on rectangular matrix is denoted by .
The flexible global Arnoldi process, recursively, generates an Forthonormal basis of the global Krylov subspace spanned by the rectangular matrices . At the end of the th iteration, we obtain a matrix relation (flexible global Arnoldi relation); that is,where , , andis an upper Hessenberg matrix.
2.2. The FGLGCRODR Algorithm
Suppose that and are given matrices satisfyingwhere is a search vector, recycled from either the previous linear system or the previous cycle, and . By minimizing the residual Fnorm,over the search space , and satisfy
Based upon what is proven in [11, 14] and using the framework discussed in [12, 32], the steps of the flexible global Arnoldi process are carried out, with the starting rectangular matrix , while holding the orthogonality to , to eventually obtain
Since , that is, , the following flexible global Arnoldilike relation holds:
At the end, we find the approximate solution by minimizing the norm , where satisfieswith .
To improve the convergence of iterative solvers, the smallest magnitude eigenvalues will be deflated. Following the idea of [12, 32], we use harmonic Ritz vectors with those eigenvalues to construct approximate invariant subspace and then the subspace for next cycles is retained. Firstly, we compute harmonic Ritz vectors , such thatwhere , and each harmonic Ritz pair satisfies
It is noted that the harmonic residual vectors satisfy
Observe that the residual . So, the residual and the harmonic residuals are in the same dimensional space . Then we can characterize the relationship by the following formula:where satisfies , , and .
Due to the relations (10) and (21) can be also expressed as
Next, we compute the reduced QRfactorization of with and and then define , assuch that . Since is orthogonal to the subspace and is orthonormal, we have
Therefore, at the end of the loop, an update is generated, scaled such that has orthonormal columns; the previous global Krylov subspace basis is dropped, and then we restart. Details of the FGLGCRODR method are summarized in Algorithm 2.


In addition, the major advantage of FGLGCRODR is that it can retain important spectral information generated during the solution of the th linear system and then exploit such information to accelerate convergence of the iterations when solving the subsequent th system. Then we detail how to recycle subspace when solving sequences of slowly changing linear systems (8) as follows (Algorithm 3).

2.3. Comments on the FGLGCRODR Method
To achieve better efficiency of the linear system solvers, we not only employ the subspace recycling at each cycle but also apply the flexible preconditioning operator at each iteration. As we know, the variable preconditioning can be construed as an inexact subspace method. If the matrix is difficult to invert, we will resort to several steps of iterative method as flexible preconditioning, so we obtain in some way an approximate result,at th (inner) iteration for the equation . In such a situation, what are the properties of the true residual obtained by using the flexible preconditioning? We will address this question in this section. Following [33, 34], we setto be the residual that results after the iterative solver has been terminated; then
Therefore, the perturbation in linear operation with can be written explicitly as
The perturbation implies that the error computed has the bound
In fact, the solution is obtained directly from the approximation subspace ; that is,indicating that the perturbation does not affect the computation of the true residual. In other words, in exact arithmeticand is orthogonal to . As a consequence, the perturbation of the linear operations only influences the goodness of and the accuracy of the approximate solution. Therefore, we deduce that, in the flexible setting, a dynamic stopping criterion or a much looser tolerance can be employed to save computation by limiting the flexible preconditioning work without sacrificing the closeness of the computed and the true residuals.
2.4. The Relationship between Two Flexible Global Methods with Subspace Recycling
As we know, FGCRODR [32] and FGMRESDR [35] are two important methods in the general category of augmented (deflated) GMREStype methods, which choose harmonic Ritz vectors as augmented (deflated) subspace. They emphasize on the solution of linear system which has only one equation. The relationship between FGCRODR and FGMRESDR has been well described by [32]. It is shown that they are algebraically equivalent as long as a collinearity condition is satisfied.In fact, FGLGCRODR is a global generalization of FGCRODR. For the sake of descriptive integrality, we also present a global generalization of FGMRESDR algorithm, referred to as FGLGMRESDR, in Algorithm 4. As for global versions, the FGLGCRODR method is also closely related to the FGLGMRESDR approach in algebraical analysis, but their numerical behavior will be slightly different. Details will be shown in Section 3.

Furthermore, we compare the main calculation cost and storage capacity for each cycle of FGLGCRODR and FGLGMRESDR in Table 1. In practical applications, . Here, and denote floatingpoint calculation counts for the preconditioning application and the linear operations, respectively.
The computational cost of harmonic Ritz vectors and “” have not been induced in Table 1 since the cost is approximately , much more less than . is referred to as the total cost of FGLGCRODR. One sees that the time complexities of two global algorithms are the same, that is, , but FGLGMRESDR requires slightly more computation per cycle than FGLGCRODR in terms of computational cost.
2.5. Application to the General Coupled Matrix Equations
FGLGCRODR substantially belongs to the family of the global Krylov subspace method, which can be also applied to solve the general coupled matrix equations of the formincluding the (coupled) Sylvester and the (coupled) Lyapunov matrix equations. The literature on solving (32) is particularly rich and still growing [36–53], due to the prominent role of such matrix equations in stability theory [54], perturbation analysis [55], control theory [56, 57], and so forth. In particular, the global subspace solvers perform better than other approaches for solving (32), especially for the coefficient matrices which are large and sparse; for more details, see [44–53, 58]. Therefore, we are interested in exploiting the FGLGCRODR method for solving (32). To do so, the linear operator can be defined as follows:where . Exploiting the operator , we can rewrite (32) aswhere . The effectiveness of FGLGCRODR for the solution of (32) is shown in Section 3.
3. Numerical Experiments
This section presents some numerical examples to illustrate the effectiveness of our algorithm for solving shifted linear system (1) and matrix equations (8) and compare the performance with the global GMRES approach (referred to as GLGMRES ) [31, 50, 59] and GLGMRESDR [60], with or without preconditioning. In each example, the flexible preconditioning is dominated by 5 steps of the global full GMRES [50].
We consider three numerical illustrations here. The first two experiments are sequences of linear systems arising from lattice QCD [2–4, 61] and the finite difference scheme of the convectiondiffusionreaction equation [62], respectively. The last one is the general coupled Sylvester matrix equations. The comparisons are mainly made in two aspects: the execution time in seconds (T (s), for short) and number of matrixvector products (mvps, for short).
All the numerical experiments are carried out on MATLAB 2017b. We choose , , and . The stopping criteria are for all the approaches, or when mvps exceed the set maximum of matrixvector multiplications (, for short). Moreover, the symbol represents that the solver does not achieve the stopping criteria within and .
Example 1. We first consider two groups of complex Lattice QCD matrices from Matrix Market (please refer to http://math.nist.gov/MatrixMarket for details). One group has seven matrices and the other has seven matrices. For each matrix , a critical parameter is provided such that with is positivereal. For each set, we take as the sequence matrix, and . The numerical computation is carried out with . The corresponding numerical results are reported in Tables 2 and 3. In addition, we also depict the residual histories for linear systems solved using iterative algorithms with or without flexible preconditioning.
As seen from Tables 2 and 3 and Figures 1 and 2 , it is found that using deflation and flexible preconditioning is conducive to improving the convergence rate. FGLGCRODR has a significant decrease in terms of mvps and T(s), which are similar to FGLGMRESDR, but performs slightly better. The GLGMRES method is less effective and does not converge in some test problems. The above results show that FGLGCRODR has the potential to improve the convergence, compared with other global Krylov subspace solvers.
Example 2. For the second case, we assess the performance of global solvers on threedimensional separable model convectiondiffusion problem in the unit cube with the homogeneous Dirichlet boundary conditions [62]:where and . For obtaining the sequence matrices, we set with and and then use the central finite difference scheme to discrete the convectiondiffusion equation with the mesh size or . Thus, two groups of complex matrices are obtained. One group has five matrices and the other has five matrices. As the definition of , we choose .
The numerical results reported in Tables 4 and 5 and Figures 3 and 4 show that FGLGMRES is slightly superior to FGLGCRODR when the mesh size . However, as the number of grid points increases, FGLGCRODR enjoys the significant decrease in the number of mvps and T(s). Meanwhile, according to Tables 4 and 5, we can see that the convergence is speeded up remarkably by considering the flexible preconditioning. Moreover, even the size of grid points increases, and the average number of mvps for FGLGCRODR does not increase.
Example 3. For the last test case, we are interested in solving the general coupled Sylvester matrix equations [45, 47]whereHere, we consider different orders. As the definition of , we choose . The corresponding numerical results of mvps and T(s) are reported in Table 6 and the residual histories are depicted in Figures 5 and 6.
As seen from Table 5 and Figures 5 and 6, it is noted that the flexible deflated GMREStype methods (FGLGCRODR and FGLGMRESDR) are found to be efficient in terms of performance test problems of different orders, while GLGMRES and FGLGMRES fail to converge in the given steps with or without variable preconditioning, while the parameter is greater than 80. The convergence is improved remarkably by considering the eigenvalue deflation technique and the flexible preconditioning. One should note that FGLGCRODR enjoys the significant decrease in the number of mvps and T(s) and performs slightly better than FGLGMRESDR method. In addition, when increases, the average number of mvps and T(s) for FGLGCRODR do not increase.
4. Conclusions
In this study, a new flexible global GCRODR method is proposed for addressing sequences of shifted linear systems and general coupled matrix equations. The proposed method does not require the righthand sides to be related and can also be compatible with general preconditioning. By performing an error analysis, we deduce that a much looser tolerance can be applied to save computation by limiting the flexible preconditioned work without sacrificing the closeness of the computed and the true residuals. Meanwhile, we also propose a global version of the FGMRESDR method and then analyze the relationship between two flexible global methods. Finally, numerical experiments show that FGLGCRODR significantly reduces the mvps, saves the runtime, and enjoys faster convergence than some other global GMREStype approaches on tough problems.
As future work, we plan to investigate the numerical properties of FGLGCRODR on largescale realistic problems. Of interest are applications related to, for example, model adaptation in Gaussian process models [63] or stochastic finite element methods [64] in three dimensions where variable preconditioning using inexact solvers has to be usually considered. Another extension of this work is a distributedmemory parallel implementation and testing using MPI + OpenMP and MPI + CUDA. Investigation will focus on implementing, testing, and comparing the runtime of the introduced FGLGCRODR on CPUs and GPUs with respect to other existent similar methods.
Data Availability
The data will be made available upon reasonable request via email to the corresponding author.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
All authors collaborated in carrying out the study and all of them read and approved the final manuscript.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (nos. 11601365 and 11801463), Project of Shandong Science and Technology Department Foundation (no. J16LI06), Foundation of National Key Laboratory of Computational Physics (no. 6142A05180203), Project of Natural Science Foundation of Shandong Province (no. ZR2016AM06), and the Scientific Research Fund of Hunan Provincial Education Department (no. 19B381).