Modified Halfspace-Relaxation Projection Methods for Solving the Split Feasibility Problem

Abstract

This paper presents modified halfspace-relaxation projection (HRP) methods for solving the split feasibility problem (SFP). Incorporating with the techniques of identifying the optimal step length with positive lower bounds, the new methods improve the efficiencies of the HRP method (Qu and Xiu (2008)). Some numerical results are reported to verify the computational preference.

1. Introduction

Let 𝐶 and 𝑄 be nonempty closed convex sets in 𝑅^𝑛 and 𝑅^𝑚, respectively, and 𝐴 an 𝑚×𝑛 real matrix. The problem, to find 𝑥∈𝐶 with 𝐴𝑥∈𝑄 if such 𝑥 exists, was called the split feasibility problem (SFP) by Censor and Elfving [1].

In this paper, we consider an equivalent reformulation [2] of the SFP: (𝑧)𝑥𝑦minimize𝑓subjectto𝑧=∈Ω, (1.1) where 𝑓(𝑧)=12‖𝐵𝑧‖²=12𝑦−𝐴𝑥²,𝐵=−𝐴𝐼,Ω=𝐶×𝑄. (1.2) For convenience, we only consider the Euclidean norm. It is obvious that 𝑓(𝑧) is convex. If 𝑧=(𝑥^𝑇,𝑦^𝑇)^𝑇∈Ω and 𝑓(𝑧)=0, then 𝑥 solves the SFP. Throughout we assume that the solution set of the SFP is nonempty. And thus the solution set of (1.1), denoted by Ω^∗, is nonempty. In addition, in this paper, we always assume that the set Ω is given by Ω=𝑧∈𝑅^𝑛+𝑚∣𝑐(𝑧)≤0, (1.3) where 𝑐∶𝑅^𝑛+𝑚→𝑅 is a convex (not necessarily differentiable) function. This representation of Ω is general enough, because any system of inequalities {𝑐_𝑗(𝑧)≤0,𝑗∈𝐽}, where 𝑐_𝑗(𝑧) are convex and 𝐽 is an arbitrary index set, can be reformulated as the single inequality 𝑐(𝑧)≤0 with 𝑐(𝑧)=sup{𝑐_𝑗(𝑧)∣𝑗∈𝐽}. For any 𝑧∈𝑅^𝑛+𝑚, at least one subgradient 𝜉∈𝜕𝑐(𝑧) can be calculated, where 𝜕𝑐(𝑧) is a subgradient of 𝑐(𝑧) at 𝑧 and is defined as follows: 𝜕𝑐(𝑧)=𝜉∈𝑅^𝑛+𝑚∣𝑐(𝑢)≥𝑐(𝑧)+(𝑢−𝑧)^𝑇𝜉,∀𝑢∈𝑅^𝑛+𝑚. (1.4) Qu and Xiu [2] proposed a halfspace-relaxation projection method to solve the convex optimization problem (1.1). Starting from any 𝑧⁰∈𝑅^𝑛×𝑅^𝑚, the HRP method iteratively updates 𝑧^𝑘 according to the formulae: 𝑧^𝑘=𝑃_Ω𝑘𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘,𝑧^𝑘+1=𝑧^𝑘−𝛾_𝑘𝑧^𝑘−𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘, (1.5)(1.6) where Ω_𝑘=𝑧∈𝑅^𝑛+𝑚∣𝑐𝑧^𝑘+𝑧−𝑧^𝑘𝑇𝜉^𝑘≤0, (1.7)𝜉^𝑘 is an element in 𝜕𝑐(𝑧^𝑘), 𝛼_𝑘=𝛾𝑙^𝑚𝑘 and 𝑚_𝑘 is the smallest nonnegative integer 𝑚 such that 𝛼_𝑘𝑧^𝑘−𝑧^𝑘𝑇∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘≤1−𝜌𝑧^𝑘−𝑧^𝑘2(),𝛾,𝜌∈0,1_𝑘=𝜃𝜌𝑧^𝑘−𝑧^𝑘2𝑧^𝑘−𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘2,𝜃∈(0,2). (1.8)(1.9) The notation 𝑃_Ω𝑘(𝑣) denotes the projection of 𝑣 onto Ω_𝑘 under the Euclidean norm, that is, 𝑃_Ω𝑘(𝑣)=argmin‖𝑢−𝑣‖∣𝑢∈Ω_𝑘. (1.10) Here the halfspace Ω_𝑘 contains the given closed convex set Ω and is related to the current iterative point 𝑧^𝑘. From the expressions of Ω_𝑘, the projection onto Ω_𝑘 is simple to be computed (for details, see Proposition 3.3). The idea to construct the halfspace Ω_𝑘 and replace 𝑃_Ω by 𝑃_Ω𝑘 is from the halfspace-relaxation projection technique presented by Fukushima [3]. This technique is often used to design algorithms (see, e.g., [2, 4, 5]) to solve the SFP. The drawback of the HRP method in [2] is that the step length 𝛾_𝑘 defined in (1.9) may be very small since lim_𝑘→∞‖𝑧^𝑘−𝑧^𝑘‖=0.

Note that the reformulation (1.1) is equivalent to a monotone variational inequality (VI): 𝑧^∗∈Ω,𝑧−𝑧^∗𝑇∇𝑓𝑧^∗≥0,∀𝑧∈Ω, (1.11) where ∇𝑓(𝑧)=𝐵^𝑇𝐵𝑧. (1.12) The forward-backward splitting method [6] and the extragradient method [7, 8] are considerably simple projection-type methods in the literature. They are applicable for solving monotone variational inequalities, especially for (1.11). For given 𝑧^𝑘, let 𝑧^𝑘=𝑃_Ω𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘. (1.13) Under the assumption 𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘≤𝜈𝑧^𝑘−𝑧^𝑘,𝜈∈(0,1), (1.14) the forward-backward (FB) splitting method generates the new iterate via 𝑧^𝑘+1=𝑃_Ω𝑧^𝑘+𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘, (1.15) while the extra-gradient (EG) method generates the new iterate by 𝑧^𝑘+1=𝑃_Ω𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘. (1.16) The forward-backward splitting method (1.15) can be rewritten as 𝑧^𝑘+1=𝑃_Ω𝑧^𝑘−𝛾_𝑘𝑧^𝑘−𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘, (1.17) where the descent direction −(𝑧^𝑘−𝑧^𝑘−𝛼_𝑘(∇𝑓(𝑧^𝑘)−∇𝑓(𝑧^𝑘))) is the same as (1.6) and the step length 𝛾_𝑘 along this direction always equals to 1. He et al. [9] proposed the modified versions of the FB method and EG method by incorporating the optimal step length 𝛾_𝑘 along the descent directions −(𝑧^𝑘−𝑧^𝑘−𝛼_𝑘(∇𝑓(𝑧^𝑘)−∇𝑓(𝑧^𝑘))) and −𝛼_𝑘∇𝑓(𝑧^𝑘), respectively. Here 𝛾_𝑘 is defined by 𝛾_𝑘=𝜃𝛾^∗_𝑘,𝜃∈(0,2),𝛾^∗_𝑘=𝑧^𝑘−𝑧^𝑘𝑇𝑧^𝑘−𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘𝑧^𝑘−𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘2. (1.18) Under the assumption (1.14), 𝛾^∗_𝑘≥1/2 is lower bounded.

This paper is to develop two kinds of modified halfspace-relaxation projection methods for solving the SFP by improving the HRP method in [2]. One is an FB type HRP method, the other is an EG type HRP method. The numerical results reported in [9] show that efforts of identifying the optimal step length usually lead to attractive numerical improvements. This fact triggers us to investigate the selection of optimal step length with positive lower bounds in the new methods to accelerate convergence. The preferences to the HRP method are verified by numerical experiments for the test problems arising in [2].

The rest of this paper is organized as follows. In Section 2, we summarize some preliminaries of variational inequalities. In Section 3, we present the new methods and provide some remarks. The selection of optimal step length of the new methods is investigated in Section 4. Then, the global convergence of the new methods is proved in Section 5. Some preliminary numerical results are reported in Section 6 to show the efficiency of the new methods, and the numerical superiority to the HRP method in [2]. Finally, some conclusions are made in Section 7.

2. Preliminaries

In the following, we state some basic concepts for the variational inequality VI(𝒮,𝐹): 𝑠^∗∈𝒮,𝑠−𝑠^∗𝑇𝐹𝑠^∗≥0,∀𝑠∈𝒮, (2.1) where 𝐹 is a mapping from 𝑅^𝑁 into 𝑅^𝑁, and 𝒮⊆𝑅^𝑁 is a nonempty closed convex set. The mapping 𝐹 is said to be monotone on 𝑅^𝑁 if (𝑠−𝑡)^𝑇(𝐹(𝑠)−𝐹(𝑡))≥0,∀𝑠,𝑡∈𝑅^𝑁. (2.2) Notice that the variational inequality VI(𝒮,𝐹) is invariant when we multiply 𝐹 by some positive scalar 𝛼. Thus VI(𝒮,𝐹) is equivalent to the following projection equation (see [10]): 𝑠=𝑃_𝒮[𝑠−𝛼𝐹(𝑠)], (2.3) that is, to solve VI(𝒮,𝐹) is equivalent to finding a zero point of the residue function 𝑒(𝑠,𝛼)∶=𝑠−𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]. (2.4) Note that 𝑒(𝑠,𝛼) is a continuous function of 𝑠 because the projection mapping is nonexpansive. The following lemma states a useful property of ‖𝑒(𝑠,𝛼)‖.

Lemma 2.1 ([4], Lemma 2.2). Let 𝐹 be a mapping from 𝑅^𝑁 into 𝑅^𝑁. For any 𝑠∈𝑅^𝑁 and 𝛼>0, we have min{1,𝛼}‖𝑒(𝑠,1)‖≤‖𝑒(𝑠,𝛼)‖≤max{1,𝛼}‖𝑒(𝑠,1)‖. (2.5)

Remark 2.2. Let 𝒮⊇𝒮 be a nonempty closed convex set and let 𝑒_𝒮(𝑠,𝛼) be defined as follows: 𝑒_𝒮(𝑠,𝛼)=𝑠−𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]. (2.6) Inequalities (2.5) still hold for 𝑒_𝒮(𝑠,𝛼).

Some fundamental inequalities are listed below without proof, see, for example, [10].

Lemma 2.3. Let 𝒮 be a nonempty closed convex set. Then the following inequalities always hold 𝑡−𝑃_𝒮(𝑡)^𝑇𝑃_𝒮(𝑡)−𝑠≥0,∀𝑡∈𝑅^𝑁𝑃,∀𝑠∈𝒮,_𝒮(𝑡)−𝑠²≤‖𝑡−𝑠‖²−𝑡−𝑃_𝒮(𝑡)²,∀𝑡∈𝑅^𝑁,𝑠∈𝒮. (2.7)(2.8)

The next lemma lists some inequalities which will be useful for the following analysis.

Lemma 2.4. Let 𝒮⊇𝒮 be a nonempty closed convex set, 𝑠^∗ a solution of the monotone VI(𝒮,𝐹) (2.1) and especially 𝐹(𝑠^∗)=0. For any 𝑠∈𝑅^𝑁 and 𝛼>0, one has 𝛼𝑠−𝑠^∗𝑇𝐹𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]≥𝛼𝑒_𝒮(𝑠,𝛼)^𝑇𝐹𝑃_𝒮[𝑠−𝛼𝐹(𝑠)],𝑠−𝑠^∗𝑇𝑒_𝒮(𝑠,𝛼)−𝛼𝐹(𝑠)−𝐹𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]≥𝑒_𝒮(𝑠,𝛼)^𝑇𝑒_𝒮(𝑠,𝛼)−𝛼𝐹(𝑠)−𝐹𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]. (2.9)(2.10)

Proof. Under the assumption that 𝐹 is monotone we have 𝛼𝐹𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]−𝛼𝐹𝑠^∗𝑇𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]−𝑠^∗≥0,∀𝑠∈𝑅^𝑁. (2.11) Using 𝐹(𝑠^∗)=0 and the notation of 𝑒_𝒮(𝑠,𝛼), from (2.11) the assertion (2.9) is proved. Setting 𝑡=𝑠−𝛼𝐹(𝑠) and 𝑠=𝑠^∗ in the inequality (2.7) and using the notation of 𝑒_𝒮(𝑠,𝛼), we obtain 𝑒_𝒮(𝑠,𝛼)−𝛼𝐹(𝑠)^𝑇𝑃_𝒮[𝑠−𝛼𝐹(𝑠)]−𝑠^∗≥0,∀𝑠∈𝑅^𝑁. (2.12) Adding (2.11) and (2.12), and using 𝐹(𝑠^∗)=0, we have (2.10). The proof is complete.

Note that the assumption 𝐹(𝑠^∗)=0 in Lemma 2.4 is reasonable. The following proposition and remark will explain this.

Proposition 2.5 ([2], Proposition 2.2). For the optimization problem (1.1), the following two statements are equivalent: (i)𝑧^∗∈Ω and 𝑓(𝑧^∗)=0,(ii)𝑧^∗∈Ω and ∇𝑓(𝑧^∗)=0.

Remark 2.6. Under the assumption that the solution set of the SFP is nonempty, if 𝑧^∗=((𝑥^∗)^𝑇,(𝑦^∗)^𝑇)^𝑇 is a solution of (1.1), then we have ∇𝑓𝑧^∗=𝐵^𝑇𝐵𝑧^∗=𝐵^𝑇𝑦^∗−𝐴𝑥^∗=0. (2.13) This point 𝑧^∗ is also the solution point of the VI (1.11).

The next lemma provides an important boundedness property of the subdifferential, see, for example, [11].

Lemma 2.7. Suppose ℎ∶𝑅^𝑁→𝑅 is a convex function, then it is subdifferentiable everywhere and its subdifferentials are uniformly bounded on any bounded subset of 𝑅^𝑁.

3. Modified Halfspace-Relaxation Projection Methods

In this section, we will propose two kinds of modified halfspace-relaxation projection methods—Algorithms 1 and 2. Algorithm 1 is an FB type HRP method and Algorithm 2 is an EG type HRP method. The relationship of these two methods is that they use the same optimal step length along different descent directions. The detailed procedures are presented as below.

The Modified Halfspace-Relaxation Projection Methods
Step 1. Let 𝛼₀>0, 0<𝜇<𝜈<1, 𝑧⁰∈𝑅^𝑛+𝑚, 𝜃∈(0,2), 𝜀>0 and 𝑘=0. (In practical computation, we suggest to take 𝜇=0.3,𝜈=0.9 and 𝜃=1.8).
Step 2. Set 𝑧^𝑘=𝑃_Ω𝑘𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘, (3.1) where Ω_𝑘 is defined in (1.7). If ‖𝑧^𝑘−𝑧^𝑘‖≤𝜀, terminate the iteration with the iterate 𝑧^𝑘=((𝑥^𝑘)^𝑇,(𝑦^𝑘)^𝑇)^𝑇, and then 𝑥^𝑘 is the approximate solution of the SFP. Otherwise, go to Step 3.
Step 3. If 𝑟_𝑘𝛼∶=^𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘𝑧^𝑘−𝑧^𝑘≤𝜈, (3.2) then set 𝑒_𝑘𝑧^𝑘,𝛼_𝑘=𝑒_Ω𝑘𝑧^𝑘,𝛼_𝑘=𝑧^𝑘−𝑧^𝑘,𝑔_𝑘𝑧^𝑘,𝛼_𝑘=𝛼_𝑘∇𝑓𝑧^𝑘,𝑑_𝑘𝑧^𝑘,𝛼_𝑘=𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘+𝑔_𝑘𝑧^𝑘,𝛼_𝑘,𝛾^∗_𝑘=𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘²,𝛾_𝑘=𝜃𝛾^∗_𝑘,𝑧^𝑘+1=𝑃_Ω𝑘𝑧^𝑘−𝛾_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘,Algorithm1∶FBtypeHRPmethod (3.3)(3.4)(3.5)(3.6) or 𝑧^𝑘+1=𝑃_Ω𝑘𝑧^𝑘−𝛾_𝑘𝑔_𝑘𝑧^𝑘,𝛼_𝑘,𝛼Algorithm2∶EGtypeHRPmethod_𝑘3∶=2𝛼_𝑘if𝑟_𝑘𝛼≤𝜇,_𝑘𝛼otherwise,_𝑘+1=𝛼_𝑘,𝑘=𝑘+1,gotoStep2. (3.7)(3.8)
Step 4. Reduce the value of 𝛼_𝑘 by 𝛼_𝑘∶=(2/3)𝛼_𝑘∗min{1,1/𝑟_𝑘}, set𝑧^𝑘=𝑃_Ω𝑘𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘andgotoStep3. (3.9)

Remark 3.1. In Step 3, if the selected 𝛼_𝑘 satisfies 0<𝛼_𝑘≤𝜈/𝐿 (𝐿 is the largest eigenvalue of the matrix 𝐵^𝑇𝐵), then from (1.12), we have 𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘≤𝛼_𝑘𝐿𝑧^𝑘−𝑧^𝑘≤𝜈𝑧^𝑘−𝑧^𝑘, (3.10) and thus Condition (3.2) is satisfied. Without loss of generality, we can assume that inf_𝑘{𝛼_𝑘}=𝛼_min>0.

Remark 3.2. By the definition of subgradient, it is clear that the halfspace Ω_𝑘 contains Ω. From the expressions of Ω_𝑘, the orthogonal projections onto Ω_𝑘 may be directly calculated and then we have the following proposition (see [3, 12]).

Proposition 3.3. For any 𝑧∈𝑅^𝑛+𝑚, 𝑃_Ω𝑘(𝑧)=𝑐𝑧𝑧−^𝑘+𝑧−𝑧^𝑘𝑇𝜉^𝑘‖𝜉^𝑘‖²𝜉^𝑘,𝑖𝑓𝑐𝑧^𝑘+𝑧−𝑧^𝑘𝑇𝜉^𝑘>0;𝑧,𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒, (3.11) where Ω_𝑘 is defined in (1.7).

Remark 3.4. For the FB type HRP method, taking 𝑧^𝑘+1=𝑧^𝑘−𝛾_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘 (3.12) as the new iterate instead of Formula (3.6) seems more applicable in practice. Since from Proposition 3.3 the projection onto Ω_𝑘 is easy to be computed, Formula (3.6) is still preferable to generate the new iterate 𝑧^𝑘+1.

Remark 3.5. The proposed methods and the HRP method in [2] can be used to solve more general convex optimization problem minimize𝑓(𝑧)subjectto𝑧∈Ω, (3.13) where 𝑓(𝑧) is a general convex function only with the property that ∇𝑓(𝑧^∗)=0 for any solution point 𝑧^∗ of (3.13), and Ω is defined in (1.3). The corresponding theoretical analysis is similar as these methods to solve (1.1).

4. The Optimal Step Length

This section concentrates on investigating the optimal step length with positive lower bounds in order to accelerate convergence of the new methods. To justify the reason of choosing the optimal step length 𝛾_𝑘 in the FB type HRP method (3.6), we start from the following general form of the FB type HRP method: 𝑧^𝑘+1_FB𝛾=𝑃_Ω𝑘𝑧^𝑘+1_𝑃𝐶𝛾, (4.1) where 𝑧^𝑘+1_𝑃𝐶𝛾=𝑧^𝑘−𝛾𝑑_𝑘𝑧^𝑘,𝛼_𝑘. (4.2)

Let Θ_𝑘FB_𝛾∶=𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1_FB𝛾−𝑧^∗2, (4.3) which measures the progress made by the FB type HRP method. Note that Θ_𝑘(FB_𝛾) is a function of the step length 𝛾. It is natural to consider maximizing this function by choosing an optimal parameter 𝛾. The solution 𝑧^∗ is not known, so we cannot maximize Θ_𝑘(FB_𝛾) directly. The following theorem gives an estimate of Θ_𝑘(FB_𝛾) which does not include the unknown solution 𝑧^∗.

Theorem 4.1. Let 𝑧^∗ be an arbitrary point in Ω^∗. If the step length in the general FB type HRP method is taken 𝛾>0, then we have Θ_𝑘𝐹𝐵_𝛾∶=𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1_𝐹𝐵𝛾−𝑧^∗2≥Υ_𝑘𝛾, (4.4) where Υ_𝑘𝛾∶=2𝛾𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝛾²𝑑_𝑘𝑧^𝑘,𝛼_𝑘². (4.5)

Proof. Since 𝑧^𝑘+1_FB(𝛾)=𝑃_Ω𝑘[𝑧^𝑘+1_𝑃𝐶(𝛾)] and 𝑧^∗∈Ω⊆Ω_𝑘, it follows from (2.8) that 𝑧^𝑘+1_FB𝛾−𝑧^∗2≤𝑧^𝑘+1_𝑃𝐶𝛾−𝑧^∗2−𝑧^𝑘+1_𝑃𝐶𝛾−𝑧^𝑘+1_FB𝛾²≤𝑧^𝑘+1_𝑃𝐶𝛾−𝑧^∗2, (4.6) and consequently Θ_𝑘FB_𝛾≥𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1_𝑃𝐶𝛾−𝑧^∗2. (4.7) Setting 𝛼=𝛼_𝑘, 𝑠=𝑧^𝑘, 𝑠^∗=𝑧^∗ and 𝒮=Ω_𝑘 in the equality (2.10) and using the notation of 𝑒_𝑘(𝑧^𝑘,𝛼_𝑘) (see (3.3)) and 𝑑_𝑘(𝑧^𝑘,𝛼_𝑘) (see (3.4)), we have 𝑧^𝑘−𝑧^∗𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘≥𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘. (4.8) Using this and (4.2), we get 𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1_𝑃𝐶𝛾−𝑧^∗2=𝑧^𝑘−𝑧^∗2−𝑧^𝑘−𝛾𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝑧^∗2=2𝛾𝑧^𝑘−𝑧^∗𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝛾²𝑑_𝑘𝑧^𝑘,𝛼_𝑘²≥2𝛾𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝛾²𝑑_𝑘𝑧^𝑘,𝛼_𝑘², (4.9) and then from (4.7) the theorem is proved.

Similarly, we start from the general form of the EG type HRP method 𝑧^𝑘+1_EG𝛾=𝑃_Ω𝑘𝑧^𝑘−𝛾𝑔_𝑘𝑧^𝑘,𝛼_𝑘. (4.10) to analyze the optimal step length in the EG type HRP method (3.7). The following theorem estimates the “progress” in the sense of Euclidean distance made by the new iterate and thus motivates us to investigate the selection of the optimal length 𝛾_𝑘 in the EG type HRP method (3.7).

Theorem 4.2. Let 𝑧^∗ be an arbitrary point in Ω^∗. If the step length in the general EG type HRP method is taken 𝛾>0, then one has Θ_𝑘𝐸𝐺_𝛾∶=𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1_𝐸𝐺𝛾−𝑧^∗2≥Υ_𝑘𝛾, (4.11) where Υ_𝑘(𝛾) is defined in (4.5) and 𝑧^𝑘+1_𝑃𝐶(𝛾) is defined in (4.2).

Proof. Since 𝑧^𝑘+1_EG(𝛾)=𝑃_Ω𝑘[𝑧^𝑘−𝛾𝑔_𝑘(𝑧^𝑘,𝛼_𝑘)] and 𝑧^∗∈Ω⊆Ω_𝑘, it follows from (2.8) that 𝑧^𝑘+1_EG𝛾−𝑧^∗2≤𝑧^𝑘−𝛾𝑔_𝑘𝑧^𝑘,𝛼_𝑘−𝑧^∗2−𝑧^𝑘−𝛾𝑔_𝑘𝑧^𝑘,𝛼_𝑘−𝑧^𝑘+1_EG𝛾², (4.12) and consequently we get Θ_𝑘EG_𝛾≥𝑧^𝑘−𝑧^∗2−𝑧^𝑘−𝑧^∗−𝛾𝑔_𝑘𝑧^𝑘,𝛼_𝑘²+𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝛾𝑔_𝑘𝑧^𝑘,𝛼_𝑘²=𝑧^𝑘−𝑧^𝑘+1_EG𝛾²+2𝛾𝑧^𝑘−𝑧^∗𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘−2𝛾𝑧^𝑘−𝑧^𝑘+1_EG𝛾^𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘. (4.13) Setting 𝛼=𝛼_𝑘, 𝑠=𝑧^𝑘, 𝑠^∗=𝑧^∗ and 𝒮=Ω_𝑘 in the equality (2.9) and using the notation of 𝑒_𝑘(𝑧^𝑘,𝛼_𝑘) and 𝑔_𝑘(𝑧^𝑘,𝛼_𝑘) (see (3.3)), we have 𝑧^𝑘−𝑧^∗𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘≥𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘. (4.14) From the above inequality, we obtain Θ_𝑘EG_𝛾≥𝑧^𝑘−𝑧^𝑘+1_EG𝛾²+2𝛾𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘−2𝛾𝑧^𝑘−𝑧^𝑘+1_EG𝛾^𝑇𝑔_𝑘𝑧^𝑘,𝛼_𝑘. (4.15) Using 𝑔_𝑘(𝑧^𝑘,𝛼_𝑘)=𝑑_𝑘(𝑧^𝑘,𝛼_𝑘)−[𝑒_𝑘(𝑧^𝑘,𝛼_𝑘)−𝛼_𝑘∇𝑓(𝑧^𝑘)] (see (3.4)), it follows that Θ_𝑘EG_𝛾≥𝑧^𝑘−𝑧^𝑘+1_EG𝛾²+2𝛾𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−2𝛾𝑧^𝑘−𝑧^𝑘+1_EG𝛾^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘, (4.16) which can be rewritten as Θ_𝑘EG_𝛾≥𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝛾𝑑_𝑘𝑧^𝑘,𝛼_𝑘²−𝛾²𝑑_𝑘𝑧^𝑘,𝛼_𝑘²+2𝛾𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘+2𝛾𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘≥Υ_𝑘𝛾+2𝛾𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘. (4.17) Now we consider the last term in the right-hand side of (4.17). Notice that 𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝑒_𝑘𝑧^𝑘,𝛼_𝑘=𝑃_Ω𝑘𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−𝑧^𝑘+1_EG𝛾. (4.18) Setting 𝑡∶=𝑧^𝑘−𝛼_𝑘∇𝑓(𝑧^𝑘), 𝑠∶=𝑧^𝑘+1_EG(𝛾) and 𝒮=Ω_𝑘 in the basic inequality (2.7) of the projection mapping and using the notation of 𝑒_𝑘(𝑧^𝑘,𝛼_𝑘), we get 𝑃_Ω𝑘𝑧^𝑘−𝛼_𝑘∇𝑓𝑧^𝑘−𝑧^𝑘+1_EG𝛾^𝑇𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘≥0, (4.19) and therefore 𝑧^𝑘−𝑧^𝑘+1_EG𝛾−𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑒_𝑘𝑧^𝑘,𝛼_𝑘−𝛼_𝑘∇𝑓𝑧^𝑘≥0. (4.20) Substituting (4.20) in (4.17), it follows that Θ_𝑘EG_𝛾≥Υ_𝑘𝛾 (4.21) and the theorem is proved.

Theorems 4.1 and 4.2 provide the basis of the selection of the optimal step length of the new methods. Note that Υ_𝑘(𝛾) is the profit-function since it is a lower-bound of the progress obtained by the new methods (both the FB type HRP method and EG type HRP method). This motivates us to maximize the profit-function Υ_𝑘(𝛾) to accelerate convergence of the new methods. Since Υ_𝑘(𝛾) a quadratic function of 𝛾, it reaches its maximum at 𝛾^∗_𝑘𝑒∶=^𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘². (4.22)

Note that under Condition (3.2), using the notation of 𝑑_𝑘(𝑧^𝑘,𝛼_𝑘) we have 𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘=𝑒_𝑘𝑧^𝑘,𝛼_𝑘²−𝛼_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘≥(1−𝜈)𝑒_𝑘𝑧^𝑘,𝛼_𝑘². (4.23) In addition, since 𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘=𝑒_𝑘𝑧^𝑘,𝛼_𝑘²−𝛼_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘≥12𝑒_𝑘𝑧^𝑘,𝛼_𝑘²−𝛼_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘+12𝛼_𝑘∇𝑓𝑧^𝑘−∇𝑓𝑧^𝑘2=12‖𝑑_𝑘𝑧^𝑘,𝛼_𝑘‖², (4.24) we have 𝛾^∗_𝑘𝑒∶=^𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘²≥12. (4.25)

From numerical point of view, it is necessary to attach a relax factor to the optimal step length 𝛾^∗_𝑘 obtained theoretically to achieve faster convergence. The following theorem concerns how to choose the relax factor.

Theorem 4.3. Let 𝑧^∗ be an arbitrary point in Ω^∗, 𝜃 a positive constant and 𝛾^∗_𝑘 defined in (4.22). For given 𝑧^𝑘∈Ω_𝑘, 𝛼_𝑘 is chosen such that Condition (3.2) is satisfied. Whenever the new iterate 𝑧^𝑘+1(𝜃𝛾^∗_𝑘) is generated by 𝑧^𝑘+1𝜃𝛾^∗_𝑘=𝑃_Ω𝑘𝑧^𝑘−𝜃𝛾^∗_𝑘𝑑_𝑘𝑧^𝑘,𝛼_𝑘𝑜𝑟𝑧^𝑘+1𝜃𝛾^∗_𝑘=𝑃_Ω𝑘𝑧^𝑘−𝜃𝛾^∗_𝑘𝑔_𝑘𝑧^𝑘,𝛼_𝑘, (4.26) we have 𝑧^𝑘+1𝜃𝛾^∗_𝑘−𝑧^∗2≤𝑧^𝑘−𝑧^∗2−𝜃(2−𝜃)(1−𝜈)2𝑒_𝑘𝑧^𝑘,𝛼_𝑘². (4.27)

Proof. From Theorems 4.1 and 4.2 we have 𝑧^𝑘−𝑧^∗2−𝑧^𝑘+1𝜃𝛾^∗_𝑘−𝑧^∗2≥Υ_𝑘𝜃𝛾^∗_𝑘. (4.28) Using (4.5), (4.23), and (4.25), we obtain Υ_𝑘𝜃𝛾^∗_𝑘=2𝜃𝛾^∗_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝜃𝛾^∗_𝑘²𝑑_𝑘𝑧^𝑘,𝛼_𝑘²=2𝜃𝛾^∗_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘−𝜃²𝛾^∗_𝑘𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘=𝛾^∗_𝑘𝜃(2−𝜃)𝑒_𝑘𝑧^𝑘,𝛼_𝑘^𝑇𝑑_𝑘𝑧^𝑘,𝛼_𝑘≥𝜃(2−𝜃)(1−𝜈)2𝑒_𝑘𝑧^𝑘,𝛼_𝑘², (4.29) and the assertion is proved.

Theorem 4.3 shows theoretically that any 𝜃∈(0,2) guarantees that the new iterate makes progress to a solution. Therefore, in practical computation, we choose 𝛾_𝑘=𝜃𝛾^∗_𝑘 with 𝜃∈(0,2) as the step length in the new methods. We need to point out that from numerical experiments, 𝜃∈[1,2) is much preferable since it leads to better numerical performance.

5. Convergence

It follows from (4.27) that for both the FB type HRP method (3.6) and the EG type HRP method (3.7), there exists a constant 𝜏>0, such that 𝑧^𝑘+1−𝑧^∗2≤𝑧^𝑘−𝑧^∗2−𝜏𝑒_𝑘𝑧^𝑘,𝛼_𝑘²,∀𝑧^∗∈Ω^∗. (5.1) The convergence result of the proposed methods in this paper is based on the following theorem.

Theorem 5.1. Let {𝑧^𝑘} be a sequence generated by the proposed method (3.6) or (3.7). Then {𝑧^𝑘} converges to a point 𝑧, which belongs to Ω^∗.

Proof. First, from (5.1) we get lim_𝑘→∞𝑒_𝑘𝑧^𝑘,𝛼_𝑘=0. (5.2) Note that 𝑒_𝑘𝑧^𝑘,𝛼_𝑘=𝑧^𝑘−𝑧^𝑘,(see(3.3)). (5.3) We have lim_𝑘→∞𝑧^𝑘−𝑧^𝑘=0. (5.4) Again, it follows from (5.1) that the sequence {𝑧^𝑘} is bounded. Let 𝑧 be a cluster point of {𝑧^𝑘} and the subsequence {𝑧^𝑘𝑗} converges to 𝑧. We are ready to show that 𝑧 is a solution point of (1.1).
First, we show that 𝑧∈Ω. Since 𝑧^𝑘𝑗∈Ω_𝑘𝑗, then by the definition of Ω_𝑘𝑗, we have 𝑐𝑧^𝑘𝑗+𝑧^𝑘𝑗−𝑧^𝑘𝑗𝑇𝜉^𝑘𝑗≤0,∀𝑗=1,2,…. (5.5) Passing onto the limit in this inequality and taking into account (5.4) and Lemma 2.7, we obtain that 𝑐(𝑧)≤0. (5.6) Hence, we conclude 𝑧∈Ω.
Next, we need to show (𝑧−𝑧)^𝑇∇𝑓(𝑧)≥0, ∀𝑧∈Ω. To do so, we first prove lim_𝑗→∞𝑒_𝑘𝑗𝑧^𝑘𝑗,1=0. (5.7) It follows from Remark 3.1 in Section 3 that inf_𝑗{𝛼_𝑘𝑗}≥inf_𝑘{𝛼_𝑘}=𝛼_min>0. Then from Lemma 2.1, we have 𝑒_𝑘𝑗𝑧^𝑘𝑗≤𝑧,1^𝑘𝑗−𝑧^𝑘𝑗min1,𝛼_𝑘𝑗, (5.8) which, together with (5.4), implies that lim_𝑗→∞𝑒_𝑘𝑗𝑧^𝑘𝑗,1≤lim_𝑗→∞𝑧^𝑘𝑗−𝑧^𝑘𝑗min1,𝛼_𝑘𝑗≤lim_𝑗→∞𝑧^𝑘𝑗−𝑧^𝑘𝑗min1,𝛼_min=0. (5.9) Setting 𝑡=𝑧^𝑘𝑗−∇𝑓(𝑧^𝑘𝑗), 𝒮=Ω_𝑘𝑗 in the inequality (2.7), for any 𝑧∈Ω⊆Ω_𝑘𝑗, we obtain 𝑧^𝑘𝑗−∇𝑓𝑧^𝑘𝑗−𝑃_Ω𝑘𝑗𝑧^𝑘𝑗−∇𝑓𝑧^𝑘𝑗𝑇𝑃_Ω𝑘𝑗𝑧^𝑘𝑗−∇𝑓𝑧^𝑘𝑗−𝑧≥0. (5.10) From the fact that 𝑒_𝑘𝑗(𝑧^𝑘𝑗,1)=𝑧^𝑘𝑗−𝑃_Ω𝑘𝑗[𝑧^𝑘𝑗−∇𝑓(𝑧^𝑘𝑗)], we have 𝑒_𝑘𝑗𝑧^𝑘𝑗,1−∇𝑓𝑧^𝑘𝑗𝑇𝑧^𝑘𝑗−𝑒_𝑘𝑗𝑧^𝑘𝑗,1−𝑧≥0,∀𝑧∈Ω, (5.11) that is, 𝑧−𝑧^𝑘𝑗𝑇∇𝑓𝑧^𝑘𝑗+𝑒_𝑘𝑗𝑧^𝑘𝑗,1^𝑇𝑧^𝑘𝑗−𝑒_𝑘𝑗𝑧^𝑘𝑗,1−𝑧+∇𝑓𝑧^𝑘𝑗≥0,∀𝑧∈Ω. (5.12) Letting 𝑗→∞, taking into account (5.7), we deduce (𝑧−𝑧)^𝑇∇𝑓(𝑧)≥0,∀𝑧∈Ω, (5.13) which implies that 𝑧∈Ω^∗. Then from (5.1), it follows that 𝑧^𝑘+1−𝑧²≤𝑧^𝑘−𝑧²−𝜏𝑒_𝑘𝑧^𝑘,𝛼_𝑘². (5.14) Together with the fact that the subsequence {𝑧^𝑘𝑗} converges to 𝑧, we can conclude that {𝑧^𝑘} converges to 𝑧. The proof is complete.

6. Numerical Results

In this section, we implement the proposed methods to solve some numerical examples arising in [2] and then report the results. To show the superiority of the new methods, we also compare them with the HRP method in [2]. The codes for implementing the proposed methods were written by Matlab 7.9.0 (R2009b) and run on an HP Compaq 6910p Notebook (2.00 GHz of Intel Core 2 Duo CPU and 2.00 GB of RAM). The stopping criterion is ‖𝑒_𝑘(𝑧^𝑘,𝛼_𝑘)‖≤𝜀.

For the new methods, we take 𝜀=10⁻¹⁰, 𝛼₀=1, 𝜇=0.3, 𝜈=0.9 and 𝜃=1.8. To compare with the HRP method and the new methods, we list the numbers of iterations, the computation times (CPU(Sec.)) and the approximate solutions in Tables 1, 2, 3, 4, 5, 6, 7, 8, and 9. For the HRP method in [2], we list the original numerical results in [2].

Table 1 

Results for Example 6.1 using the HRP method in [2].

Table 2 

Results for Example 6.1 using FB type HRP method.

Table 3 

Results for Example 6.1 using EG type HRP method.

Table 4 

Results for Example 6.2 using the HRP method in [2].

Table 5 

Results for Example 6.2 using FB type HRP method.

Table 6 

Results for Example 6.2 using EG type HRP method.

Table 7 

Results for Example 6.3 using the HRP method in [2].

Table 8 

Results for Example 6.3 using FB type HRP method.

Table 9 

Results for Example 6.3 using EG type HRP method.

Example 6.1 (a convex feasibility problem). Let 𝐶={𝑥∈𝑅³∣𝑥²₂+𝑥²₃−4≤0}, 𝑄={𝑥∈𝑅³∣𝑥₃−1−𝑥²₁≤0}. Find some point 𝑥 in 𝐶∩𝑄.
Obviously this example can be regarded as an SFP with 𝐴=𝐼.

For Example 6.1, it is easy to verify that the point (1,1,1,1,1,1)^𝑇 is a solution of (1.1). Therefore, the FB type and EG type HRP method only use 0 iteration when we choose the starting point 𝑧⁰=(1,1,1,1,1,1)^𝑇. While applying the HRP method in [2] to solve Example 6.1 and choosing the same starting point, the number of iterations is 67. This is the original numerical result listed in Table 1 of [2].

Example 6.2 (a split feasibility problem). Let 𝐴=^2−13425₂₀₂,𝐶={𝑥∈𝑅³∣𝑥₁+𝑥²₂+2𝑥₃≤0}, 𝑄={𝑥∈𝑅³∣𝑥²₁+𝑥₂−𝑥₃≤0}. Find some point 𝑥∈𝐶 with 𝐴𝑥∈𝑄.

Example 6.3 (a nonlinear programming problem). Consider the problem Minimize𝑓(𝑧)=^𝑛_𝑖=1𝑧²_𝑖subjectto_𝑖≠𝑗𝑧²_𝑖−𝑧_𝑗−𝑗≤0,𝑗=1,…,𝑛. (6.1) This example is a general nonlinear programming problem not the reformulation (1.1) for the SFP. Notice that it has a unique solution 𝑧^∗=(0,…,0)^𝑇 and ∇𝑓(𝑧^∗)=2𝑧^∗=(0,…,0)^𝑇. Then from Remark 3.5 in Section 3, the proposed methods and the HRP method in [2] can be used to find its solution.
The computational preferences to the HRP method (1.5)-(1.6) are revealed clearly in Tables 1–9. The numerical results demonstrate that the selection of optimal step length in both the FB type HRP method and the EG type HRP method reduces considerable computational load of the HRP method in [2].

7. Conclusions

In this paper we consider the split feasibility problem, which is a special case of the multiple-sets split feasibility problem [13–15]. With some new strategies for determining the optimal step length, this paper improves the HRP method in [2] and thus develops modified halfspace-relaxation projection methods for solving the split feasibility problem. Compared to the HRP method in [2], the new methods reduce the number of iterations moderately with little additional computation.

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant no. 11001053) and the Research Award Fund for Outstanding Young Teachers in Southeast University.