Numerical Methods for Solving Variational Inequalities and Complementarity Problems
View this Special IssueResearch Article  Open Access
Modified HalfspaceRelaxation Projection Methods for Solving the Split Feasibility Problem
Abstract
This paper presents modified halfspacerelaxation projection (HRP) methods for solving the split feasibility problem (SFP). Incorporating with the techniques of identifying the optimal step length with positive lower bounds, the new methods improve the efficiencies of the HRP method (Qu and Xiu (2008)). Some numerical results are reported to verify the computational preference.
1. Introduction
Let and be nonempty closed convex sets in and , respectively, and an real matrix. The problem, to find with if such exists, was called the split feasibility problem (SFP) by Censor and Elfving [1].
In this paper, we consider an equivalent reformulation [2] of the SFP: where For convenience, we only consider the Euclidean norm. It is obvious that is convex. If and , then solves the SFP. Throughout we assume that the solution set of the SFP is nonempty. And thus the solution set of (1.1), denoted by , is nonempty. In addition, in this paper, we always assume that the set is given by where is a convex (not necessarily differentiable) function. This representation of is general enough, because any system of inequalities , where are convex and is an arbitrary index set, can be reformulated as the single inequality with . For any , at least one subgradient can be calculated, where is a subgradient of at and is defined as follows: Qu and Xiu [2] proposed a halfspacerelaxation projection method to solve the convex optimization problem (1.1). Starting from any , the HRP method iteratively updates according to the formulae: where is an element in , and is the smallest nonnegative integer such that The notation denotes the projection of onto under the Euclidean norm, that is, Here the halfspace contains the given closed convex set and is related to the current iterative point . From the expressions of , the projection onto is simple to be computed (for details, see Proposition 3.3). The idea to construct the halfspace and replace by is from the halfspacerelaxation projection technique presented by Fukushima [3]. This technique is often used to design algorithms (see, e.g., [2, 4, 5]) to solve the SFP. The drawback of the HRP method in [2] is that the step length defined in (1.9) may be very small since .
Note that the reformulation (1.1) is equivalent to a monotone variational inequality (VI): where The forwardbackward splitting method [6] and the extragradient method [7, 8] are considerably simple projectiontype methods in the literature. They are applicable for solving monotone variational inequalities, especially for (1.11). For given , let Under the assumption the forwardbackward (FB) splitting method generates the new iterate via while the extragradient (EG) method generates the new iterate by The forwardbackward splitting method (1.15) can be rewritten as where the descent direction is the same as (1.6) and the step length along this direction always equals to . He et al. [9] proposed the modified versions of the FB method and EG method by incorporating the optimal step length along the descent directions and , respectively. Here is defined by Under the assumption (1.14), is lower bounded.
This paper is to develop two kinds of modified halfspacerelaxation projection methods for solving the SFP by improving the HRP method in [2]. One is an FB type HRP method, the other is an EG type HRP method. The numerical results reported in [9] show that efforts of identifying the optimal step length usually lead to attractive numerical improvements. This fact triggers us to investigate the selection of optimal step length with positive lower bounds in the new methods to accelerate convergence. The preferences to the HRP method are verified by numerical experiments for the test problems arising in [2].
The rest of this paper is organized as follows. In Section 2, we summarize some preliminaries of variational inequalities. In Section 3, we present the new methods and provide some remarks. The selection of optimal step length of the new methods is investigated in Section 4. Then, the global convergence of the new methods is proved in Section 5. Some preliminary numerical results are reported in Section 6 to show the efficiency of the new methods, and the numerical superiority to the HRP method in [2]. Finally, some conclusions are made in Section 7.
2. Preliminaries
In the following, we state some basic concepts for the variational inequality VI: where is a mapping from into , and is a nonempty closed convex set. The mapping is said to be monotone on if Notice that the variational inequality VI is invariant when we multiply by some positive scalar . Thus VI is equivalent to the following projection equation (see [10]): that is, to solve VI is equivalent to finding a zero point of the residue function Note that is a continuous function of because the projection mapping is nonexpansive. The following lemma states a useful property of .
Lemma 2.1 ([4], Lemma 2.2). Let be a mapping from into . For any and , we have
Remark 2.2. Let be a nonempty closed convex set and let be defined as follows: Inequalities (2.5) still hold for .
Some fundamental inequalities are listed below without proof, see, for example, [10].
Lemma 2.3. Let be a nonempty closed convex set. Then the following inequalities always hold
The next lemma lists some inequalities which will be useful for the following analysis.
Lemma 2.4. Let be a nonempty closed convex set, a solution of the monotone VI (2.1) and especially . For any and , one has
Proof. Under the assumption that is monotone we have Using and the notation of , from (2.11) the assertion (2.9) is proved. Setting and in the inequality (2.7) and using the notation of , we obtain Adding (2.11) and (2.12), and using , we have (2.10). The proof is complete.
Note that the assumption in Lemma 2.4 is reasonable. The following proposition and remark will explain this.
Proposition 2.5 ([2], Proposition 2.2). For the optimization problem (1.1), the following two statements are equivalent: (i) and ,(ii) and .
Remark 2.6. Under the assumption that the solution set of the SFP is nonempty, if is a solution of (1.1), then we have This point is also the solution point of the VI (1.11).
The next lemma provides an important boundedness property of the subdifferential, see, for example, [11].
Lemma 2.7. Suppose is a convex function, then it is subdifferentiable everywhere and its subdifferentials are uniformly bounded on any bounded subset of .
3. Modified HalfspaceRelaxation Projection Methods
In this section, we will propose two kinds of modified halfspacerelaxation projection methods—Algorithms 1 and 2. Algorithm 1 is an FB type HRP method and Algorithm 2 is an EG type HRP method. The relationship of these two methods is that they use the same optimal step length along different descent directions. The detailed procedures are presented as below.
The Modified HalfspaceRelaxation Projection Methods
Step 1. Let , , , , and . (In practical computation, we suggest to take and ).
Step 2. Set
where is defined in (1.7). If , terminate the iteration with the iterate , and then is the approximate solution of the SFP. Otherwise, go to Step 3.
Step 3. If
then set
or
Step 4. Reduce the value of by ,
Remark 3.1. In Step 3, if the selected satisfies ( is the largest eigenvalue of the matrix ), then from (1.12), we have and thus Condition (3.2) is satisfied. Without loss of generality, we can assume that .
Remark 3.2. By the definition of subgradient, it is clear that the halfspace contains . From the expressions of , the orthogonal projections onto may be directly calculated and then we have the following proposition (see [3, 12]).
Proposition 3.3. For any , where is defined in (1.7).
Remark 3.4. For the FB type HRP method, taking as the new iterate instead of Formula (3.6) seems more applicable in practice. Since from Proposition 3.3 the projection onto is easy to be computed, Formula (3.6) is still preferable to generate the new iterate .
Remark 3.5. The proposed methods and the HRP method in [2] can be used to solve more general convex optimization problem where is a general convex function only with the property that for any solution point of (3.13), and is defined in (1.3). The corresponding theoretical analysis is similar as these methods to solve (1.1).
4. The Optimal Step Length
This section concentrates on investigating the optimal step length with positive lower bounds in order to accelerate convergence of the new methods. To justify the reason of choosing the optimal step length in the FB type HRP method (3.6), we start from the following general form of the FB type HRP method: where
Let which measures the progress made by the FB type HRP method. Note that is a function of the step length . It is natural to consider maximizing this function by choosing an optimal parameter . The solution is not known, so we cannot maximize directly. The following theorem gives an estimate of which does not include the unknown solution .
Theorem 4.1. Let be an arbitrary point in . If the step length in the general FB type HRP method is taken , then we have where
Proof. Since and , it follows from (2.8) that and consequently Setting , , and in the equality (2.10) and using the notation of (see (3.3)) and (see (3.4)), we have Using this and (4.2), we get and then from (4.7) the theorem is proved.
Similarly, we start from the general form of the EG type HRP method to analyze the optimal step length in the EG type HRP method (3.7). The following theorem estimates the “progress” in the sense of Euclidean distance made by the new iterate and thus motivates us to investigate the selection of the optimal length in the EG type HRP method (3.7).
Theorem 4.2. Let be an arbitrary point in . If the step length in the general EG type HRP method is taken , then one has where is defined in (4.5) and is defined in (4.2).
Proof. Since and , it follows from (2.8) that and consequently we get Setting , , and in the equality (2.9) and using the notation of and (see (3.3)), we have From the above inequality, we obtain Using (see (3.4)), it follows that which can be rewritten as Now we consider the last term in the righthand side of (4.17). Notice that Setting , and in the basic inequality (2.7) of the projection mapping and using the notation of , we get and therefore Substituting (4.20) in (4.17), it follows that and the theorem is proved.
Theorems 4.1 and 4.2 provide the basis of the selection of the optimal step length of the new methods. Note that is the profitfunction since it is a lowerbound of the progress obtained by the new methods (both the FB type HRP method and EG type HRP method). This motivates us to maximize the profitfunction to accelerate convergence of the new methods. Since a quadratic function of , it reaches its maximum at
Note that under Condition (3.2), using the notation of we have In addition, since we have
From numerical point of view, it is necessary to attach a relax factor to the optimal step length obtained theoretically to achieve faster convergence. The following theorem concerns how to choose the relax factor.
Theorem 4.3. Let be an arbitrary point in , a positive constant and defined in (4.22). For given , is chosen such that Condition (3.2) is satisfied. Whenever the new iterate is generated by we have
Proof. From Theorems 4.1 and 4.2 we have Using (4.5), (4.23), and (4.25), we obtain and the assertion is proved.
Theorem 4.3 shows theoretically that any guarantees that the new iterate makes progress to a solution. Therefore, in practical computation, we choose with as the step length in the new methods. We need to point out that from numerical experiments, is much preferable since it leads to better numerical performance.
5. Convergence
It follows from (4.27) that for both the FB type HRP method (3.6) and the EG type HRP method (3.7), there exists a constant , such that The convergence result of the proposed methods in this paper is based on the following theorem.
Theorem 5.1. Let be a sequence generated by the proposed method (3.6) or (3.7). Then converges to a point , which belongs to .
Proof. First, from (5.1) we get
Note that
We have
Again, it follows from (5.1) that the sequence is bounded. Let be a cluster point of and the subsequence converges to . We are ready to show that is a solution point of (1.1).
First, we show that . Since , then by the definition of , we have
Passing onto the limit in this inequality and taking into account (5.4) and Lemma 2.7, we obtain that
Hence, we conclude .
Next, we need to show , . To do so, we first prove
It follows from Remark 3.1 in Section 3 that . Then from Lemma 2.1, we have
which, together with (5.4), implies that
Setting , in the inequality (2.7), for any , we obtain
From the fact that , we have
that is,
Letting , taking into account (5.7), we deduce
which implies that . Then from (5.1), it follows that
Together with the fact that the subsequence converges to , we can conclude that converges to . The proof is complete.
6. Numerical Results
In this section, we implement the proposed methods to solve some numerical examples arising in [2] and then report the results. To show the superiority of the new methods, we also compare them with the HRP method in [2]. The codes for implementing the proposed methods were written by Matlab 7.9.0 (R2009b) and run on an HP Compaq 6910p Notebook (2.00 GHz of Intel Core 2 Duo CPU and 2.00 GB of RAM). The stopping criterion is .
For the new methods, we take , , , and . To compare with the HRP method and the new methods, we list the numbers of iterations, the computation times (CPU(Sec.)) and the approximate solutions in Tables 1, 2, 3, 4, 5, 6, 7, 8, and 9. For the HRP method in [2], we list the original numerical results in [2].









Example 6.1 (a convex feasibility problem). Let , . Find some point in .
Obviously this example can be regarded as an SFP with .
For Example 6.1, it is easy to verify that the point is a solution of (1.1). Therefore, the FB type and EG type HRP method only use iteration when we choose the starting point . While applying the HRP method in [2] to solve Example 6.1 and choosing the same starting point, the number of iterations is 67. This is the original numerical result listed in Table 1 of [2].
Example 6.2 (a split feasibility problem). Let , . Find some point with .
Example 6.3 (a nonlinear programming problem). Consider the problem This example is a general nonlinear programming problem not the reformulation (1.1) for the SFP. Notice that it has a unique solution