Residual Iterative Method for Solving Absolute Value Equations

Abstract

We suggest and analyze a residual iterative method for solving absolute value equations 𝐴𝑥−|𝑥|=𝑏 where 𝐴∈𝑅^𝑛×𝑛, 𝑏∈𝑅^𝑛 are given and 𝑥∈𝑅^𝑛 is unknown, using the projection technique. We also discuss the convergence of the proposed method. Several examples are given to illustrate the implementation and efficiency of the method. Comparison with other methods is also given. Results proved in this paper may stimulate further research in this fascinating field.

1. Introduction

The residual methods were proposed for solving large sparse systems of linear equations 𝐴𝑥=𝑏,(1.1) where 𝐴∈𝑅^𝑛×𝑛 is a positive definite matrix and 𝑥,𝑏∈𝑅^𝑛. Paige and Saunders [1] minimized the residual norm over the Krylov subspace and proposed an algorithm for solving indefinite systems. Saad and Schultz [2] used Arnoldi process and suggested generalized minimal residual method which minimize norm of the residual at each step. The residual methods have been studied extensively [3–5].

We show that the Petrov-Galerkin process can be extended for solving absolute value equations of the form: 𝐴𝑥−|𝑥|=𝑏,(1.2) where 𝐴∈𝑅^𝑛×𝑛, 𝑏∈𝑅^𝑛. Here |𝑥| is the vector in 𝑅^𝑛 with absolute values of the components of 𝑥 and 𝑥∈𝑅^𝑛, is unknown. The absolute value equations (1.1) were investigated extensively in [6]. It was Managasarian [7, 8], who proved that the absolute value equations (1.2) are equivalent to the linear complementarity problems. This equivalent formulation was used by Managasarian [7, 8] to solve the absolute value equations. We would like to remark that the complementarity problems are also equivalent to the variational inequalities. Thus, we conclude that the absolute value equations are equivalent to the variational inequalities. There are several methods for solving the variational inequalities; see Noor [9–11], Noor et al. [12, 13] and the references therein. To the best our knowledge, this alternative equivalent formulation has not exploited up to now. This is another direction for future direction. We hope that this interlink among these fields may lead to discover novel and innovative techniques for solving the absolute value equations and related optimization problems. Noor et al. [14, 15] have suggested some iterative methods for solving absolute value equation (1.2) using minimization technique with symmetric positive definite matrix. For more details, see [3, 4, 6–12, 14–19].

In this paper, we suggest and analyse residual iterative method for solving absolute value equations (1.2) using projection technique. Our method is easy to implement. We discuss the convergence of the residual method for nonsymmetric positive definite matrices.

We denote by 𝐾 and 𝐿 the search subspace and the constraints subspace, respectively, and let 𝑚 be their dimension and 𝑥₀∈𝑅^𝑛 an initial guess. A projection method onto the subspace 𝐾 and orthogonal to 𝐿 is a process to find an approximate solution 𝑥∈𝑅^𝑛 to (1.2) by imposing the Petrov-Galerkin conditions that 𝑥 belong to affine space 𝑥₀+𝐾 such that the new residual vector orthogonal to 𝐿, that is, find𝑥∈𝑥₀+𝐾suchthat𝑏−(𝐴−𝐷(𝑥))𝑥⟂𝐿,(1.3) where 𝐷(𝑥) is diagonal matrix corresponding to sign(𝑥). For different choices of the subspace 𝐿, we have different iterative methods. Here we use the constraint space 𝐿=(𝐴−𝐷(𝑥))𝐾. The residual method approximates the solution of (1.2) by the vector 𝑥∈𝑥₀+𝐾that minimizes the norm of residual.

The inner product is denoted by ⟨⋅,⋅⟩ in the 𝑛-dimensional Euclidean space 𝑅^𝑛. For 𝑥∈𝑅^𝑛, sign(𝑥) will denote a vector with components equal to 1,0,−1 depending on whether the corresponding component of 𝑥 is positive, zero, or negative. The diagonal matrix 𝐷(𝑥) corresponding to sign(𝑥) is defined as𝐷(𝑥)=𝜕|𝑥|=diag(sign(𝑥)),(1.4) where 𝜕|𝑥| represent the generalized Jacobean of |𝑥| based on a subgradient [20, 21].

We denote the following by 𝑎𝑐==⟨⟨𝐶𝐶𝑣𝑣₁₁,𝐶𝑣₁⟩,𝑑=⟨𝐶𝑣₂,𝐶𝑣₂⟩,𝑝,𝐶𝑣₂⟩,₁𝑝=𝑏−𝐴𝑥_𝑘+||𝑥_𝑘||,𝐶𝑣₁=⟨𝑏−𝐶𝑥_𝑘,𝐶𝑣₁⟩,₂=𝑏−𝐴𝑥_𝑘+||𝑥_𝑘||,𝐶𝑣₂=⟨𝑏−𝐶𝑥_𝑘,𝐶𝑣₂⟩,(1.5) where 0≠𝑣₁, 𝑣₂∈𝑅^𝑛, and 𝐶=𝐴−𝐷(𝑥_𝑘). We consider 𝐴 such that 𝐶 is a positive definite matrix. We remark that 𝐷(𝑥_𝑘)𝑥_𝑘=|𝑥_𝑘|.

2. Residual Iterative Method

Consider the iterative scheme of the type: 𝑥_𝑘+1=𝑥_𝑘+𝛼𝑣₁+𝛽𝑣₂,0≠𝑣₁,𝑣₂∈𝑅^𝑛,𝑘=0,1,2,….(2.1) These vectors can be chosen by different ways. To derive residual method for solving absolute value equations in the first step, we choose the subspace𝐾₁=span𝑣₁,𝐿₁=span𝐶𝑣₁,𝑥₀=𝑥_𝑘.(2.2) For 𝐷(𝑥̃_𝑘+1)=𝐷(𝑥_𝑘), we write the residual in the following form: 𝑏−𝐴𝑥̃_𝑘+1+||𝑥̃_𝑘+1||=𝑏−𝐴−𝐷𝑥̃_𝑘+1𝑥̃_𝑘+1=𝑏−𝐴−𝐷𝑥_𝑘𝑥̃_𝑘+1=𝑏−𝐶𝑥̃_𝑘+1.(2.3) From (1.3) and (2.3), we calculate 𝑥̃_𝑘+1∈𝑥_𝑘+𝐾₁suchthat𝑏−𝐶𝑥̃_𝑘+1⟂𝐿₁;(2.4) that is, we find the approximate solution by the iterative scheme 𝑥̃_𝑘+1=𝑥_𝑘+𝛼𝑣₁.(2.5) Now, we rewrite (2.4) in the inner product as 𝑏−𝐶𝑥̃_𝑘+1,𝐶𝑣₁=0;(2.6) from the above discussion, we have ⟨𝑏−𝐶𝑥_𝑘−𝛼𝐶𝑣₁,𝐶𝑣₁⟩==𝑝⟨𝑏₁−𝐶𝑥_𝑘,𝐶𝑣₁⟩−𝛼⟨𝐶𝑣₁,𝐶𝑣₁⟩−𝑎𝛼=0,(2.7) from which we have 𝛼=𝑝¹𝑎.(2.8) The next step is to choose the subspace 𝐾₂=span𝑣₂,𝐿₂=span𝐶𝑣₂,𝑥₀=𝑥̃_𝑘+1,(2.9) and to find the approximate solution 𝑥_𝑘+1 such that 𝑥_𝑘+1∈𝑥̃_𝑘+1+𝐾₂suchthat𝑏−𝐶𝑥_𝑘+1⟂𝐿₂,(2.10) where 𝑏−𝐴𝑥_𝑘+1𝑥_𝑘+1=𝑥̃_𝑘+1+𝛽𝑣₂,+||𝑥_𝑘+1||=𝑏−𝐶𝑥_𝑘+1,𝐷𝑥_𝑘+1=𝐷𝑥_𝑘.(2.11) Rewriting (2.10) in terms of the inner product, we have𝑏−𝐶𝑥_𝑘+1,𝐶𝑣₂=0.(2.12) Thus, we have𝑏−𝐶𝑥_𝑘+1,𝐶𝑣₂==⟨⟨𝑏𝑏−−𝐶𝐶𝑥𝑥_𝑘𝑘−𝛼𝐶𝑣₁−𝛽𝐶𝑣₂,𝐶𝑣₂⟩=𝑝₂,𝐶𝑣₂⟩−𝛼⟨𝐶𝑣₁,𝐶𝑣₂⟩−𝛽⟨𝐶𝑣₂,𝐶𝑣₂⟩−𝑐𝛼−𝑑𝛽=0.(2.13) From (2.8) and (2.13), we obtain𝛽=𝑎𝑝²−𝑐𝑝₁𝑎𝑑.(2.14)

We remark that one can choose 𝑣₁=𝑟_𝑘 and 𝑣₂ in different ways. However, we consider the case 𝑣₂=𝑠_𝑘 (𝑠_𝑘 is given in Algorithm 2.1).

Based upon the above discussion, we suggest and analyze the following iterative method for solving the absolute value equations (1.2) and this is the main motivation of this paper.

Algorithm 2.1. Choose an initial guess 𝑥₀∈𝑅^𝑛,For 𝑘=0,1,2,… until convergence do𝑟_𝑘=𝑏−𝐴𝑥_𝑘+|𝑥_𝑘|𝑔_𝑘=(𝐴−𝐷(𝑥_𝑘))^𝑇(𝐴𝑥_𝑘−|𝑥_𝑘|−𝑏)𝐻_𝑘=((𝐴−𝐷(𝑥_𝑘))⁻¹(𝐴−𝐷(𝑥_𝑘)))^𝑇𝑠_𝑘=−𝐻_𝑘𝑔_𝑘If ‖𝑟_𝑘‖=0, then stop; else𝛼_𝑘=𝑝₁/𝑎,𝛽_𝑘=(𝑎𝑝₂−𝑐𝑝₁)/𝑎𝑑Set 𝑥_𝑘+1=𝑥_𝑘+𝛼_𝑘𝑟_𝑘+𝛽_𝑘𝑠_𝑘 If ‖𝑥_𝑘+1−𝑥_𝑘‖<10⁻⁶then stopEnd ifEnd for 𝑘.

If 𝛽=0, then Algorithm 2.1 reduces to minimal residual method; see [2, 5, 21, 22]. For the convergence analysis of Algorithm 2.1, we need the following result.

Theorem 2.2. Let {𝑥_𝑘} and {𝑟_𝑘} be generated by Algorithm 2.1; if 𝐷(𝑥_𝑘+1)=𝐷(𝑥_𝑘), then ‖‖𝑟_𝑘‖‖²−‖‖𝑟_𝑘+1‖‖²=𝑝²¹𝑎+𝑎𝑝²−𝑐𝑝₁²𝑎²𝑑,(2.15) where 𝑟_𝑘+1=𝑏−𝐴𝑥_𝑘+1+|𝑥_𝑘+1| and 𝐷(𝑥_𝑘+1)=diag(sign(𝑥_𝑘+1)).

Proof. Using (2.1), we obtain 𝑟_𝑘+1==𝑏𝑏−−𝐴𝐴𝑥−_𝑘+1𝐷+||𝑥𝑥_𝑘+1𝑘+1||𝑥_𝑘+1=𝑏−𝐴−𝐷𝑥_𝑘𝑥_𝑘+1=𝑏−𝐴−𝐷𝑥_𝑘𝑥_𝑘−𝛼𝐴−𝐷𝑥_𝑘𝑣₁−𝛽𝐴−𝐷𝑥_𝑘𝑣₂=𝑏−𝐴𝑥_𝑘+||𝑥_𝑘||−𝛼𝐶𝑣₁−𝛽𝐶𝑣₂=𝑟_𝑘−𝛼𝐶𝑣₁−𝛽𝐶𝑣₂.(2.16) Now consider ‖‖𝑟_𝑘+1‖‖²==⟨𝑟𝑟_𝑘+1𝑘,𝑟_𝑘+1=⟨𝑟_𝑘−𝛼𝐶𝑣₁−𝛽𝐶𝑣₂,𝑟_𝑘−𝛼𝐶𝑣₁−𝛽𝐶𝑣₂⟩=‖‖𝑟_𝑘,𝑟_𝑘⟩−2𝛼⟨𝑟_𝑘,𝐶𝑣₁⟩−2𝛼𝛽⟨𝐶𝑣₁,𝐶𝑣₂⟩−2𝛽⟨𝑟_𝑘,𝐶𝑣₂⟩+𝛼²⟨𝐶𝑣₁,𝐶𝑣₁⟩+𝛽²⟨𝐶𝑣₂,𝐶𝑣₂⟩‖‖²−2𝛼𝑝₁+2𝑐𝛼𝛽−2𝛽𝑝₂+𝑎𝛼²+𝛽²𝑑.(2.17) From (2.8), (2.14), and (2.17), we have ‖‖𝑟_𝑘‖‖²−‖‖𝑟_𝑘+1‖‖²=𝑝²¹𝑎+𝑎𝑝²−𝑐𝑝₁²𝑎²𝑑,(2.18) the required result (2.15).

Since 𝑝²¹/𝑎+(𝑎𝑝₂−𝑐𝑝₁)²/𝑎²𝑑≥0, so from (2.18) we have ‖‖𝑟_𝑘‖‖²−‖‖𝑟_𝑘+1‖‖²=𝑝²¹𝑎+𝑎𝑝²−𝑐𝑝₁²𝑎²𝑑≥0.(2.19) From (2.19) we have ‖𝑟_𝑘+1‖²≤‖𝑟_𝑘‖². For any arbitrary vectors 0≠𝑣₁,𝑣₂∈𝑅^𝑛, 𝛼,𝛽 are defined by (2.8), and (2.14) minimizes norm of the residual.

We now consider the convergence criteria of Algorithm 2.1, and it is the motivation of our next result.

Theorem 2.3. If 𝐶 is a positive definite matrix, then the approximate solution obtained from Algorithm 2.1 converges to the exact solution of the absolute value equations (1.2).

Proof. From (2.15), we have ‖‖𝑟_𝑘‖‖²−‖‖𝑟_𝑘+1‖‖²≥𝑝²¹𝑎=⟨𝑟^𝑘,𝐶𝑟_𝑘⟩²⟨𝐶𝑟_𝑘,𝐶𝑟_𝑘⟩≥𝜆^2min‖‖𝑟_𝑘‖‖⁴𝜆^2max‖‖𝑟_𝑘‖‖²=𝜆^2min𝜆^2max‖‖𝑟_𝑘‖‖².(2.20) This means that the sequence ‖𝑟_𝑘‖² is decreasing and bounded. Thus the above sequence is convergent which implies that the left-hand side tends to zero. Hence ‖𝑟_𝑘‖² tends to zero, and the proof is complete.

3. Numerical Results

To illustrate the implementation and efficiency of the proposed method, we consider the following examples. All the experiments are performed with Intel(R) Core(TM) 2 × 2.1 GHz, 1 GB RAM, and the codes are written in Mat lab 7.

Example 3.1. Consider the ordinary differential equation: 𝑑²𝑥𝑑𝑡²−|𝑥|=1−𝑡²,0≤𝑡≤1,𝑥(0)=−1𝑥(1)=0.(3.1) We discredited the above equation using finite difference method to obtain the system of absolute value equations of the type: 𝐴𝑥−|𝑥|=𝑏,(3.2) where the system matrix 𝐴 of size 𝑛=10 is given by 𝑎_𝑖,𝑗=⎧⎪⎨⎪⎩−102,214,2,ffooothrre𝑗rw=𝑗𝑗is𝑖==,e.𝑖𝑖+−11,,𝑖𝑖==12,,23,,……,,𝑛𝑛,−1,(3.3) The exact solution is 𝑥=.1915802528sin𝑡−4cos𝑡+3−𝑡²−1.462117157𝑒_−𝑡,𝑥<0,−0.5378828428𝑒^𝑡+1+𝑡²,𝑥>0.(3.4) In Figure 1, we compare residual method with Noor et al. [14, 15]. The residual iterative method, minimization method [14], and the iterative method [10] solve (3.1) in 51, 142, and 431 iterations, respectively. For the next two examples, we interchange 𝑣₁,𝑣₂ with each other as Algorithm 2.1 converges for nonzero vectors 𝑣₁,𝑣₂∈𝑅^𝑛.

Figure 1 

Example 3.2 (see [17]). We first chose a random 𝐴 from a uniform distribution on [−10,10], then we chose a random 𝑥 from a uniform distribution on [−1, 1]. Finally we computed 𝑏=𝐴𝑥−|𝑥|. We ensured that the singular values of each 𝐴 exceeded 1 by actually computing the minimum singular value and rescaling 𝐴 by dividing it by the minimum singular value multiplied by a random number in the interval [0, 1]. The computational results are given in Table 1.

Table 1 

In Table 1, GNM and RIM denote generalized Newton method [17] and residual iterative method. From Table 1 we conclude that residual method for solving absolute value equations (1.2) is more effective.

Example 3.3 (see [23]). Consider random matrix 𝐴 and 𝑏 in Mat lab code as 𝑛=input𝐴("dr=𝑅𝑏aimn=𝑅=dernr_∗a(as"ninsoddtnat((eo𝑛𝑛"f,,,1𝑛m0)))a;;;trix𝐴=");Run^∗eye(𝑛),(3.5) with random initial guess. The comparison between the residual iterative method and the Yong method [23] is presented in Table 2.

Table 2 

In Table 2 TOC denotes time taken by CPU. Note that for large problem sizes the residual iterative method converges faster than the Yong method [23].

4. Conclusions

In this paper, we have used the projection technique to suggest an iterative method for solving the absolute value equations. The convergence analysis of the proposed method is also discussed. Some examples are given to illustrate the efficiency and implementation of the new iterative method. The extension of the proposed iterative method for solving the general absolute value equation of the form 𝐴𝑥+𝐵|𝑥|=𝑏 for suitable matrices is an open problem. We have remarked that the variational inequalities are also equivalent to the absolute value equations. This equivalent formulation can be used to suggest and analyze some iterative methods for solving the absolute value equations. It is an interesting and challenging problem to consider the variational inequalities for solving the absolute value equations.

Acknowledgments

This research is supported by the Visiting Professor Program of the King Saud University, Riyadh, Saudi Arabia, and Research Grant no. KSU.VPP.108. The authors are also grateful to Dr. S. M. Junaid Zaidi, Rector, COMSATS Institute of Information Technology, Pakistan, for providing the excellent research facilities.