A Hybrid Gradient-Projection Algorithm for Averaged Mappings in Hilbert Spaces

Abstract

It is well known that the gradient-projection algorithm (GPA) is very useful in solving constrained convex minimization problems. In this paper, we combine a general iterative method with the gradient-projection algorithm to propose a hybrid gradient-projection algorithm and prove that the sequence generated by the hybrid gradient-projection algorithm converges in norm to a minimizer of constrained convex minimization problems which solves a variational inequality.

1. Introduction

Let 𝐻 be a real Hilbert space and 𝐶 a nonempty closed and convex subset of 𝐻. Consider the following constrained convex minimization problem: minimize_𝑥∈𝐶𝑓(𝑥),(1.1) where 𝑓∶𝐶→ℝ is a real-valued convex and continuously Fréchet differentiable function. The gradient ∇𝑓 satisfies the following Lipschitz condition: ‖∇𝑓(𝑥)−∇𝑓(𝑦)‖≤𝐿‖𝑥−𝑦‖,∀𝑥,𝑦∈𝐶,(1.2) where 𝐿>0. Assume that the minimization problem (1.1) is consistent, and let 𝑆 denote its solution set.

It is well known that the gradient-projection algorithm is very useful in dealing with constrained convex minimization problems and has extensively been studied ([1–5] and the references therein). It has recently been applied to solve split feasibility problems [6–10]. Levitin and Polyak [1] consider the following gradient-projection algorithm: 𝑥_𝑛+1∶=Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,𝑛≥0.(1.3) Let {𝜆_𝑛}^∞𝑛=0 satisfy 0<liminf_𝑛→∞𝜆_𝑛≤limsup_𝑛→∞𝜆_𝑛<2𝐿.(1.4) It is proved that the sequence {𝑥_𝑛} generated by (1.3) converges weakly to a minimizer of (1.1).

Xu proved that under certain appropriate conditions on {𝛼_𝑛} and {𝜆_𝑛} the sequence {𝑥_𝑛} defined by the following relaxed gradient-projection algorithm: 𝑥_𝑛+1=1−𝛼_𝑛𝑥_𝑛+𝛼_𝑛Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,𝑛≥0,(1.5) converges weakly to a minimizer of (1.1) [11].

Since the Lipschitz continuity of the gradient of 𝑓 implies that it is indeed inverse strongly monotone (ism) [12, 13], its complement can be an averaged mapping. Recall that a mapping 𝑇 is nonexpansive if and only if it is Lipschitz with Lipschitz constant not more than one, that a mapping is an averaged mapping if and only if it can be expressed as a proper convex combination of the identity mapping and a nonexpansive mapping, and that a mapping 𝑇 is said to be 𝜈-inverse strongly monotone if and only if ⟨𝑥−𝑦,𝑇𝑥−𝑇𝑦⟩≥𝜈‖𝑇𝑥−𝑇𝑦‖²forall𝑥,𝑦∈𝐻, where the number 𝜈>0. Recall also that the composite of finitely many averaged mappings is averaged. That is, if each of the mappings {𝑇_𝑖}^𝑁𝑖=1 is averaged, then so is the composite 𝑇₁⋯𝑇_𝑁 [14]. In particular, an averaged mapping is a nonexpansive mapping [15]. As a result, the GPA can be rewritten as the composite of a projection and an averaged mapping which is again an averaged mapping.

Generally speaking, in infinite-dimensional Hilbert spaces, GPA has only weak convergence. Xu [11] provided a modification of GPA so that strong convergence is guaranteed. He considered the following hybrid gradient-projection algorithm: 𝑥_𝑛+1=𝜃_𝑛ℎ𝑥_𝑛+1−𝜃_𝑛Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛.(1.6)

It is proved that if the sequences {𝜃_𝑛} and {𝜆_𝑛} satisfy appropriate conditions, the sequence {𝑥_𝑛} generated by (1.6) converges in norm to a minimizer of (1.1) which solves the variational inequality 𝑥^∗∈𝑆,⟨(𝐼−ℎ)𝑥^∗,𝑥−𝑥^∗⟩≥0,𝑥∈𝑆.(1.7)

On the other hand, Ming Tian [16] introduced the following general iterative algorithm for solving the variational inequality 𝑥_𝑛+1=𝛼_𝑛𝛾𝑓𝑥_𝑛+𝐼−𝜇𝛼_𝑛𝐹𝑇𝑥_𝑛,𝑛≥0,(1.8) where 𝐹 is a 𝜅-Lipschitzian and 𝜂-strongly monotone operator with 𝜅>0, 𝜂>0 and 𝑓 is a contraction with coefficient 0<𝛼<1. Then, he proved that if {𝛼_𝑛} satisfying appropriate conditions, the {𝑥_𝑛} generated by (1.8) converges strongly to the unique solution of variational inequality ⟨(𝜇𝐹−𝛾𝑓)𝑥̃,𝑥̃−𝑧⟩≤0,𝑧∈Fix(𝑇).(1.9)

In this paper, motivated and inspired by the research work in this direction, we will combine the iterative method (1.8) with the gradient-projection algorithm (1.3) and consider the following hybrid gradient-projection algorithm: 𝑥_𝑛+1=𝜃_𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜇𝜃_𝑛𝐹Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,𝑛≥0.(1.10)

We will prove that if the sequence {𝜃_𝑛} of parameters and the sequence {𝜆_𝑛} of parameters satisfy appropriate conditions, then the sequence {𝑥_𝑛} generated by (1.10) converges in norm to a minimizer of (1.1) which solves the variational inequality (𝑉𝐼)𝑥^∗∈𝑆,⟨(𝜇𝐹−𝛾ℎ)𝑥^∗,𝑥−𝑥^∗⟩≥0,∀𝑥∈𝑆,(1.11) where 𝑆 is the solution set of the minimization problem (1.1).

2. Preliminaries

This section collects some lemmas which will be used in the proofs for the main results in the next section. Some of them are known; others are not hard to derive.

Throughout this paper, we write 𝑥_𝑛⇀𝑥 to indicate that the sequence {𝑥_𝑛} converges weakly to 𝑥, 𝑥_𝑛→𝑥 implies that {𝑥_𝑛} converges strongly to 𝑥. 𝜔_𝑤(𝑥_𝑛)∶={𝑥∶∃𝑥_𝑛𝑗⇀𝑥} is the weak 𝜔-limit set of the sequence {𝑥_𝑛}^∞𝑛=1.

Lemma 2.1 (see [17]). Assume that {𝑎_𝑛}^∞𝑛=0 is a sequence of nonnegative real numbers such that 𝑎_𝑛+1≤1−𝛾_𝑛𝑎_𝑛+𝛾_𝑛𝛿_𝑛+𝛽_𝑛,𝑛≥0,(2.1) where {𝛾_𝑛}^∞𝑛=0 and {𝛽_𝑛}^∞𝑛=0 are sequences in [0,1] and {𝛿_𝑛}^∞𝑛=0 is a sequence in ℝ such that(i)∑^∞𝑛=0𝛾_𝑛=∞; (ii)either limsup_𝑛→∞𝛿_𝑛≤0 or ∑^∞𝑛=0𝛾_𝑛|𝛿_𝑛|<∞;(iii)∑^∞𝑛=0𝛽_𝑛<∞. Then lim_𝑛→∞𝑎_𝑛=0.

Lemma 2.2 (see [18]). Let 𝐶 be a closed and convex subset of a Hilbert space 𝐻, and let 𝑇∶𝐶→𝐶 be a nonexpansive mapping with Fix𝑇≠∅. If {𝑥_𝑛}^∞𝑛=1 is a sequence in 𝐶 weakly converging to 𝑥 and if {(𝐼−𝑇)𝑥_𝑛}^∞𝑛=1 converges strongly to y, then (𝐼−T)𝑥=𝑦.

Lemma 2.3. Let 𝐻 be a Hilbert space, and let 𝐶 be a nonempty closed and convex subset of 𝐻. ℎ∶𝐶→𝐶 a contraction with coefficient 0<𝜌<1, and 𝐹∶𝐶→𝐶 a 𝜅-Lipschitzian continuous operator and 𝜂-strongly monotone operator with 𝜅,𝜂>0. Then, for 0<𝛾<𝜇𝜂/𝜌, ⟨𝑥−𝑦,(𝜇𝐹−𝛾ℎ)𝑥−(𝜇𝐹−𝛾ℎ)𝑦⟩≥(𝜇𝜂−𝛾𝜌)‖𝑥−𝑦‖²,∀𝑥,𝑦∈𝐶.(2.2) That is, 𝜇𝐹−𝛾ℎ is strongly monotone with coefficient 𝜇𝜂−𝛾𝜌.

Lemma 2.4. Let 𝐶 be a closed subset of a real Hilbert space 𝐻, given 𝑥∈𝐻 and 𝑦∈𝐶. Then, 𝑦=𝑃_𝐶𝑥 if and only if there holds the inequality ⟨𝑥−𝑦,𝑦−𝑧⟩≥0,∀𝑧∈𝐶.(2.3)

3. Main Results

Let 𝐻 be a real Hilbert space, and let 𝐶 be a nonempty closed and convex subset of 𝐻 such that 𝐶±𝐶⊂𝐶. Assume that the minimization problem (1.1) is consistent, and let 𝑆 denote its solution set. Assume that the gradient ∇𝑓 satisfies the Lipschitz condition (1.2). Since 𝑆 is a closed convex subset, the nearest point projection from 𝐻 onto 𝑆 is well defined. Recall also that a contraction on 𝐶 is a self-mapping of 𝐶 such that ‖ℎ(𝑥)−ℎ(𝑦)‖≤𝜌‖𝑥−𝑦‖,forall𝑥,𝑦∈𝐶, where 𝜌∈[0,1) is a constant. Let 𝐹 be a 𝜅-Lipschitzian and 𝜂-strongly monotone operator on 𝐶 with 𝜅,𝜂>0. Denote by Π the collection of all contractions on 𝐶, namely, Π={ℎ∶ℎisacontractionon𝐶}.(3.1) Now given ℎ∈Π with 0<𝜌<1, 𝑠∈(0,1). Let 0<𝜇<2𝜂/𝜅²,   0<𝛾<𝜇(𝜂−(𝜇𝜅²)/2)/𝜌=𝜏/𝜌. Assume that 𝜆_𝑠 with respect to 𝑠 is continuous and, in addition, 𝜆_𝑠∈[𝑎,𝑏]⊂(0,2/𝐿). Consider a mapping 𝑋_𝑠 on 𝐶 defined by 𝑋_𝑠(𝑥)=𝑠𝛾ℎ(𝑥)+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓(𝑥),𝑥∈𝐶.(3.2) It is easy to see that 𝑋_𝑠 is a contraction. Setting 𝑉_𝑠∶=Proj_𝐶(𝐼−𝜆_𝑠∇𝑓). It is obvious that 𝑉_𝑠 is a nonexpansive mapping. We can rewrite 𝑋_𝑠(𝑥) as 𝑋_𝑠(𝑥)=𝑠𝛾ℎ(𝑥)+(𝐼−𝑠𝜇𝐹)𝑉_𝑠(𝑥).(3.3) First observe that for 𝑠∈(0,1), we can get ‖‖(𝐼−𝑠𝜇𝐹)𝑉_𝑠(𝑥)−(𝐼−s𝜇𝐹)𝑉_𝑠(𝑦)‖‖²=‖‖𝑉_𝑠(𝑥)−𝑉_𝑠(𝑦)−𝑠𝜇𝐹𝑉_𝑠(𝑥)−𝐹𝑉_𝑠(𝑦)‖‖²+=𝑠‖‖₂𝑉_𝑠(𝑥)−𝑉_𝑠(𝑦)‖‖²−2𝑠𝜇⟨𝑉_𝑠(𝑥)−𝑉_𝑠(𝑦),𝐹𝑉_𝑠(𝑥)−𝐹𝑉_𝑠(𝑦)⟩𝜇²‖‖𝐹𝑉_𝑠(𝑥)−𝐹𝑉_𝑠(𝑦)‖‖²≤‖𝑥−𝑦‖²−2𝑠𝜇𝜂‖‖𝑉_𝑠(𝑥)−𝑉_𝑠(𝑦)‖‖²+𝑠²𝜇²𝜅²‖‖𝑉_𝑠(𝑥)−𝑉_𝑠(𝑦)‖‖²≤1−𝑠𝜇2𝜂−𝑠𝜇𝜅²‖𝑥−𝑦‖²≤1−𝑠𝜇2𝜂−𝑠𝜇𝜅²2²‖𝑥−𝑦‖²≤1−𝑠𝜇𝜂−𝜇𝜅²2²‖𝑥−𝑦‖²=(1−𝑠𝜏)²‖𝑥−𝑦‖².(3.4) Indeed, we have ‖‖𝑋_𝑠(𝑥)−𝑋_𝑠(𝑦)‖‖≤=𝑠𝛾‖‖‖𝑠ℎ𝛾(ℎ𝑥()𝑥−)+ℎ((𝑦𝐼)−‖𝑠+𝜇‖‖𝐹()𝐼𝑉−_𝑠(𝑠𝑥𝜇)𝐹−)𝑠𝑉𝛾_𝑠ℎ(𝑦)−(𝐼−𝑠𝜇𝐹)𝑉_𝑠(𝑦)‖‖≤=𝑠(1𝛾𝜌−‖𝑠𝑥(𝜏−−𝑦‖𝛾𝜌+))(‖1𝑥−−𝑠𝜏𝑦)‖‖.𝑥−𝑦‖(𝑥)−(𝐼−𝑠𝜇𝐹)𝑉_𝑠(𝑦)‖‖(3.5) Hence, 𝑋_𝑠 has a unique fixed point, denoted 𝑥_𝑠, which uniquely solves the fixed-point equation 𝑥_𝑠=𝑠𝛾ℎ𝑥_𝑠+(𝐼−𝑠𝜇𝐹)𝑉_𝑠𝑥_𝑠.(3.6) The next proposition summarizes the properties of {𝑥_𝑠}.

Proposition 3.1. Let 𝑥_𝑠 be defined by (3.6).(i){𝑥_𝑠}isboundedfor𝑠∈(0,(1/𝜏)). (ii)lim_𝑠→0‖𝑥_𝑠−Proj_𝐶(𝐼−𝜆_𝑠∇𝑓)(𝑥_𝑠)‖=0. (iii)𝑥_𝑠definesacontinuouscurvefrom(0,1/𝜏)into𝐻.

Proof. (i) Take a 𝑥∈𝑆, then we have ‖‖𝑥_𝑠−𝑥=‖‖‖‖=(𝐼‖‖−𝑠𝛾𝑠ℎ𝜇𝐹𝑥)_𝑠Pr+oj(_𝐶𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−𝑥‖‖+𝑠𝛾ℎ𝑥_𝑠𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥≤(1−𝑠𝜏)‖‖𝑥_𝑠−𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥‖‖≤(1−𝑠𝜏)‖‖𝑥_𝑠−𝑥‖‖+𝑠‖‖𝛾ℎ𝑥_𝑠−𝜇𝐹𝑥‖‖−𝑥‖‖+𝑠𝛾𝜌‖‖𝑥_𝑠−𝑥‖‖+𝑠‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖.(3.7) It follows that ‖‖𝑥_𝑠−𝑥‖‖≤‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖𝜏−𝛾𝜌.(3.8) Hence, {𝑥_𝑠} is bounded.
(ii) By the definition of {𝑥_𝑠}, we have ‖‖𝑥_𝑠−Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠‖‖=‖‖𝑠−𝛾Pℎro𝑥j_𝑠𝐶+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠=𝑠‖‖𝛾ℎ𝑥_𝑠𝐼−𝜆_𝑠∇𝑓𝑥_𝑠‖‖−𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_s‖‖⟶0,(3.9){𝑥_𝑠} is bounded, so are {ℎ(𝑥_𝑠)} and {𝐹Proj_𝐶(𝐼−𝜆_𝑠∇𝑓)(𝑥_𝑠)}.
(iii) Take 𝑠, 𝑠₀∈(0,1/𝜏), and we have ‖‖𝑥=_𝑠−‖‖𝑠𝑥𝛾ℎ_𝑠0‖‖𝑥_𝑠−𝐼−𝑠₀+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−𝑠₀𝛾ℎ𝑥_𝑠0≤‖‖𝑠−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠0‖‖+‖‖𝐼−𝑠₀𝛾ℎ𝑥_𝑠+𝑠₀𝛾ℎ𝑥_𝑠−ℎ𝑥_𝑠0‖‖+‖‖(𝐼−𝑠𝜇𝜇𝐹𝐹)PrPorjo_𝐶j_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠−𝐼−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠0‖‖≤‖‖𝑠−𝑠₀𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−𝐼−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖+‖‖𝐼−𝑠₀𝛾ℎ𝑥_𝑠+𝑠₀𝛾ℎ𝑥_𝑠−ℎ𝑥_𝑠0‖‖+‖‖(𝐼−𝑠𝜇𝜇𝐹𝐹)PrPorjo_𝐶j_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠−𝐼−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠0‖‖+‖‖(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖≤||𝑠−𝑠₀𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠−𝐼−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖+||𝜆_𝑠||𝛾‖‖ℎ𝑥_𝑠‖‖+𝑠₀𝛾𝜌‖‖𝑥_𝑠−𝑥_𝑠0‖‖+1−𝑠₀𝜏‖‖𝑥_𝑠−𝑥_𝑠0‖‖+‖‖𝑠𝜇−𝐹𝜆Pr_𝑠0o||‖‖j_𝐶∇𝑓𝑥_𝑠‖‖=||𝑠−𝑠₀𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠−𝑠₀𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖+||𝜆_𝑠||𝛾‖‖ℎ𝑥_𝑠‖‖+𝑠₀𝛾𝜌‖‖𝑥_𝑠−𝑥_𝑠0‖‖+1−𝑠₀𝜏‖‖𝑥_𝑠−𝑥_𝑠0‖‖=𝛾‖‖ℎ−𝑥𝜆_𝑠0𝑠||‖‖∇𝑓𝑥_𝑠‖‖+||𝑠−𝑠₀||‖‖𝜇𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖+𝑠₀‖‖+𝜇‖‖𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖||𝑠−𝑠₀||𝛾𝜌‖‖𝑥_𝑠−𝑥_𝑠0‖‖+1−𝑠₀𝜏‖‖𝑥_𝑠−𝑥_𝑠0‖‖+||𝜆_𝑠−𝜆_𝑠0||‖‖∇𝑓𝑥_𝑠‖‖.(3.10) Therefore, ‖‖𝑥_𝑠−𝑥_𝑠0‖‖≤𝛾‖‖ℎ𝑥^𝑠‖‖+𝜇‖‖𝐹Proj_𝐶𝐼−𝜆_𝑠0∇𝑓𝑥_𝑠‖‖𝑠₀(𝜏−𝛾𝜌)||𝑠−𝑠⁰+‖‖∇𝑓𝑥_𝑠||‖‖𝑠₀(𝜏−𝛾𝜌)||𝜆^𝑠−𝜆_𝑠0||.(3.11) Therefore, 𝑥_𝑠→𝑥_𝑠0 as 𝑠→𝑠₀. This means 𝑥_𝑠 is continuous.

Our main result in the following shows that {𝑥_𝑠} converges in norm to a minimizer of (1.1) which solves some variational inequality.

Theorem 3.2. Assume that {𝑥_𝑠} is defined by (3.6), then 𝑥_𝑠 converges in norm as 𝑠→0 to a minimizer of (1.1) which solves the variational inequality ⟨(𝜇𝐹−𝛾ℎ)𝑥^∗,𝑥̃−𝑥^∗⟩≥0,∀𝑥̃∈𝑆.(3.12) Equivalently, we have Proj_𝑠(𝐼−(𝜇𝐹−𝛾ℎ))𝑥^∗=𝑥^∗.

Proof. It is easy to see that the uniqueness of a solution of the variational inequality (3.12). By Lemma 2.3, 𝜇𝐹−𝛾ℎ is strongly monotone, so the variational inequality (3.12) has only one solution. Let 𝑥^∗∈𝑆 denote the unique solution of (3.12).
To prove that 𝑥_𝑠→𝑥^∗(𝑠→0), we write, for a given 𝑥̃∈𝑆, 𝑥_𝑠−𝑥=̃=𝑠𝑠𝛾𝛾ℎℎ𝑥𝑥_𝑠𝑠+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−𝑥̃−(𝐼−𝑠𝜇𝐹−)𝜇P𝐹ro𝑥j_𝐶̃+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠𝐼−𝜆_𝑠∇𝑓(𝑥̃).(3.13) It follows that ‖‖𝑥_𝑠−𝑥̃‖‖²+=𝑠(𝐼𝛾−ℎ𝑠𝜇𝑥𝐹_𝑠)−Pr𝜇o𝐹j_𝐶𝑥̃,𝑥_𝑠−𝑥̃≤(1−𝑠𝜏)‖‖𝑥_𝑠𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓(𝑥̃),𝑥_𝑠−𝑥̃−𝑥̃‖‖²+𝑠𝛾ℎ𝑥_𝑠−𝜇𝐹𝑥̃,𝑥_𝑠−𝑥̃.(3.14) Hence, ‖‖𝑥_𝑠−𝑥̃‖‖²≤1𝜏𝛾ℎ𝑥^𝑠−𝜇𝐹𝑥̃,𝑥_𝑠≤1−𝑥̃𝜏𝛾𝜌‖‖𝑥^𝑠−𝑥̃‖‖²+⟨𝛾ℎ(𝑥̃)−𝜇𝐹𝑥̃,𝑥_𝑠−𝑥̃⟩.(3.15) To derive that ‖‖𝑥_𝑠−𝑥̃‖‖²≤1𝜏−𝛾𝜌⟨𝛾ℎ(𝑥̃)−𝜇𝐹𝑥̃,𝑥^𝑠−𝑥̃⟩.(3.16) Since {𝑥_𝑠} is bounded as 𝑠→0, we see that if {𝑠_𝑛} is a sequence in (0,1) such that 𝑠_𝑛→0 and 𝑥_𝑠𝑛⇀𝑥, then by (3.16), 𝑥_𝑠𝑛→𝑥. We may further assume that 𝜆_𝑠𝑛→𝜆∈[0,2/𝐿] due to condition (1.4). Notice that Proj_𝐶(𝐼−𝜆∇𝑓) is nonexpansive. It turns out that ‖‖𝑥_𝑠𝑛≤−‖‖P𝑥ro_𝑠𝑛j_𝐶(𝐼−𝜆∇𝑓)𝑥_𝑠𝑛‖‖≤‖‖𝑥_𝑠𝑛−Proj_𝐶𝐼−𝜆_𝑠𝑛∇𝑓𝑥_𝑠𝑛‖‖+‖‖Proj_𝐶𝐼−𝜆_𝑠𝑛∇𝑓𝑥_𝑠𝑛−Proj_𝐶(𝐼−𝜆∇𝑓)𝑥_𝑠𝑛‖‖=‖‖𝑥_𝑠𝑛−Proj_𝐶𝐼−𝜆_𝑠𝑛∇𝑓𝑥_𝑠𝑛‖‖+‖‖𝜆−𝜆_𝑠𝑛∇𝑓𝑥_𝑠𝑛‖‖−Proj_𝐶𝐼−𝜆_𝑠𝑛∇𝑓𝑥_𝑠𝑛‖‖+||𝜆−𝜆_𝑠𝑛||‖‖∇𝑓𝑥_𝑠𝑛‖‖.(3.17) From the boundedness of {𝑥_𝑠} and lim_𝑠→0‖Proj_𝐶(𝐼−𝜆_𝑠∇𝑓)𝑥_𝑠−𝑥_𝑠‖=0, we conclude that lim_𝑛→∞‖‖𝑥_𝑠𝑛−Proj_𝐶(𝐼−𝜆∇𝑓)𝑥_𝑠𝑛‖‖=0.(3.18) Since 𝑥_𝑠𝑛⇀𝑥, by Lemma 2.2, we obtain 𝑥=Proj_𝐶(𝐼−𝜆∇𝑓)𝑥.(3.19) This shows that 𝑥∈𝑆.
We next prove that 𝑥 is a solution of the variational inequality (3.12). Since 𝑥_𝑠=𝑠𝛾ℎ𝑥_𝑠+(𝐼−𝑠𝜇𝐹)Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠,(3.20) we can derive that (=𝜇𝐹−−1𝛾ℎ)𝑥_𝑠𝑠𝐼−Proj^𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠+𝜇𝐹𝑥_𝑠−𝐹Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠.(3.21) Therefore, for 𝑥̃∈𝑆, =−(1𝜇𝐹−𝛾ℎ)𝑥_𝑠,𝑥_𝑠−𝑥̃𝑠𝐼−Proj^𝐶𝐼−𝜆_𝑠+𝜇𝐹𝑥_𝑠∇𝑓𝑥_𝑠−𝐼−Proj_𝐶𝐼−𝜆_𝑠∇𝑓(𝑥̃),𝑥_𝑠−𝑥̃≤𝜇𝐹𝑥_𝑠−𝐹Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠,𝑥_𝑠−𝑥̃−𝐹Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠,𝑥_𝑠−𝑥̃.(3.22) Since Proj_𝐶(𝐼−𝜆_𝑠∇𝑓) is nonexpansive, we obtain that 𝐼−Proj_𝐶(𝐼−𝜆_𝑠∇𝑓) is monotone, that is, 𝐼−Proj_𝐶𝐼−𝜆_𝑠∇𝑓𝑥_𝑠−𝐼−Proj_𝐶𝐼−𝜆_𝑠∇𝑓(𝑥̃),𝑥_𝑠−𝑥̃≥0.(3.23) Taking the limit through 𝑠=𝑠_𝑛→0 ensures that 𝑥 is a solution to (3.12). That is to say (𝜇𝐹−𝛾ℎ)𝑥,𝑥−𝑥̃≤0.(3.24) Hence 𝑥=𝑥^∗ by uniqueness. Therefore, 𝑥_𝑠→𝑥^∗ as 𝑠→0. The variational inequality (3.12) can be written as ⟨(𝐼−𝜇𝐹+𝛾ℎ)𝑥^∗−𝑥^∗,𝑥̃−𝑥^∗⟩≤0,∀𝑥̃∈𝑆.(3.25) So, by Lemma 2.4, it is equivalent to the fixed-point equation 𝑃_𝑆(𝐼−𝜇𝐹+𝛾ℎ)𝑥^∗=𝑥^∗.(3.26)

Taking 𝐹=𝐴, 𝜇=1 in Theorem 3.2, we get the following

Corollary 3.3. We have that {𝑥_𝑠} converges in norm as 𝑠→0 to a minimizer of (1.1) which solves the variational inequality ⟨(𝐴−𝛾ℎ)𝑥^∗,𝑥̃−𝑥^∗⟩≥0,∀𝑥̃∈𝑆.(3.27) Equivalently, we have Proj_𝑠(𝐼−(𝐴−𝛾ℎ))𝑥^∗=𝑥^∗.

Taking 𝐹=𝐼, 𝜇=1, 𝛾=1 in Theorem 3.2, we get the following.

Corollary 3.4. Let 𝑧_𝑠∈𝐻 be the unique fixed point of the contraction 𝑧↦𝑠ℎ(𝑧)+(1−𝑠)Proj_𝐶(𝐼−𝜆_𝑠∇𝑓)(𝑧). Then, {𝑧_𝑠} converges in norm as 𝑠→0 to the unique solution of the variational inequality ⟨(𝐼−ℎ)𝑥^∗,𝑥̃−𝑥^∗⟩≥0,∀𝑥̃∈𝑆.(3.28)

Finally, we consider the following hybrid gradient-projection algorithm, 𝑥⁰𝑥∈𝐶arbitrarily,_𝑛+1=𝜃_𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜇𝜃_𝑛𝐹Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,∀𝑛≥0.(3.29) Assume that the sequence {𝜆_𝑛}^∞𝑛=0 satisfies the condition (1.4) and, in addition, that the following conditions are satisfied for {𝜆_𝑛}^∞𝑛=0 and {𝜃_𝑛}^∞𝑛=0⊂[0,1]:(i)𝜃_𝑛→0; (ii)∑^∞𝑛=0𝜃_𝑛=∞; (iii)∑^∞𝑛=0|𝜃_𝑛+1−𝜃_𝑛|<∞; (iv)∑^∞𝑛=0|𝜆_𝑛+1−𝜆_𝑛|<∞.

Theorem 3.5. Assume that the minimization problem (1.1) is consistent and the gradient ∇𝑓 satisfies the Lipschitz condition (1.2). Let {𝑥_𝑛} be generated by algorithm (3.29) with the sequences {𝜃_𝑛} and {𝜆_𝑛} satisfying the above conditions. Then, the sequence {𝑥_𝑛} converges in norm to 𝑥^∗ that is obtained in Theorem 3.2.

Proof. (1) The sequence {𝑥_𝑛}^∞𝑛=0 is bounded. Setting 𝑉_𝑛∶=Proj_𝐶𝐼−𝜆_𝑛∇𝑓.(3.30) Indeed, we have, for 𝑥∈𝑆, ‖‖𝑥_𝑛+1−𝑥‖‖==‖‖‖‖𝜃𝜃_𝑛𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝑥‖‖≤1−𝛾𝜃ℎ_𝑛𝑥_𝑛−𝜇𝐹𝑥+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥‖‖=1−𝜃_𝑛𝜏‖‖𝑥_𝑛−𝑥‖‖+𝜃_𝑛𝜌𝛾‖‖𝑥_𝑛−𝑥‖‖+𝜃_𝑛‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖≤max‖‖(𝜏𝑥_𝑛−𝛾𝜌)‖‖𝑥_𝑛−𝑥‖‖+𝜃_𝑛‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖−𝑥‖‖,1𝜏−𝛾𝜌‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖,∀𝑛≥0.(3.31) By induction, ‖‖𝑥_𝑛−𝑥‖‖≤max‖‖𝑥₀−𝑥‖‖,‖‖𝛾ℎ𝑥−𝜇𝐹𝑥‖‖𝜏−𝛾𝜌.(3.32) In particular, {𝑥_𝑛}^∞𝑛=0 is bounded.
(2) We prove that ‖𝑥_𝑛+1−𝑥_𝑛‖→0 as 𝑛→∞. Let 𝑀 be a constant such that 𝑀>maxsup_𝑛≥0𝛾‖‖ℎ𝑥_𝑛‖‖,sup_{𝜅,𝑛≥0}𝜇‖‖𝐹𝑉_𝜅𝑥_𝑛‖‖,sup_𝑛≥0‖‖∇𝑓𝑥_𝑛‖‖.(3.33) We compute ‖‖𝑥=‖‖_𝑛+1𝜃−_𝑛𝑥_𝑛‖‖=‖‖𝜃_𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝜃_𝑛−1𝛾ℎ𝑥_𝑛−1−𝐼−𝜇𝜃_𝑛−1𝐹𝑉_𝑛−1𝑥_𝑛−1‖‖𝛾ℎ𝑥_𝑛−ℎ𝑥_𝑛−1+𝛾𝜃_𝑛−𝜃_𝑛−1ℎ𝑥_𝑛−1+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛=−‖‖𝜃_𝑛𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−1+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−1−𝐼−𝜇𝜃_𝑛−1𝐹𝑉_𝑛−1𝑥_𝑛−1‖‖𝛾ℎ𝑥_𝑛−ℎ𝑥_𝑛−1+𝛾𝜃_𝑛−𝜃_𝑛−1ℎ𝑥_𝑛−1+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−1+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−1−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛−1𝑥_𝑛−1≤+𝜃_𝑛𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛−1𝑥_𝑛−1−𝐼−𝜇𝜃_𝑛−1𝐹𝑉_𝑛−1𝑥_𝑛−1‖‖+‖‖𝛾𝑉𝜌_𝑛‖‖𝑥_𝑛−𝑥_𝑛−1‖‖+𝛾||𝜃_𝑛−𝜃_𝑛−1||‖‖ℎ𝑥_𝑛−1‖‖+1−𝜃_𝑛𝜏‖‖𝑥_𝑛−𝑥_𝑛−1‖‖≤𝜃_𝑛𝑥_𝑛−1−𝑉_𝑛−1𝑥_𝑛−1‖‖+𝜇||𝜃_𝑛−𝜃_𝑛−1||‖‖𝐹𝑉_𝑛−1𝑥_𝑛−1‖‖+‖‖𝛾𝑉𝜌_𝑛‖‖𝑥_𝑛−𝑥_𝑛−1‖‖+𝑀||𝜃_𝑛−𝜃_𝑛−1||+1−𝜃_𝑛𝜏‖‖𝑥_𝑛−𝑥_𝑛−1‖‖=1−𝜃𝑥_𝑛−1𝑛−V_𝑛−1𝑥_𝑛−1‖‖+𝑀||𝜃_𝑛−𝜃_𝑛−1||‖‖𝑉(𝜏−𝛾𝜌)‖‖𝑥_𝑛−𝑥_𝑛−1‖‖+2𝑀||𝜃_𝑛−𝜃_𝑛−1||+‖‖𝑉_𝑛𝑥_𝑛−1−𝑉_𝑛−1𝑥_𝑛−1‖‖,(3.34)_𝑛𝑥_𝑛−1−𝑉_𝑛−1𝑥_𝑛−1‖‖=≤‖‖‖‖Pr𝐼oj−_𝐶𝜆𝐼_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛−1−Proj_𝐶𝐼−𝜆_𝑛−1∇𝑓𝑥_𝑛−1‖‖=||𝜆_𝑛∇𝑓𝑥_𝑛−1−𝐼−𝜆_𝑛−1∇𝑓𝑥_𝑛−1‖‖≤𝑀||−𝜆_𝑛𝜆_𝑛−1||‖‖∇𝑓𝑥_𝑛−1‖‖−𝜆_𝑛−1||.(3.35) Combining (3.34) and (3.35), we can obtain ‖‖𝑥_𝑛+1−𝑥_𝑛‖‖≤1−(𝜏−𝛾𝜌)𝜃_𝑛‖‖𝑥_𝑛−𝑥_𝑛−1‖‖+2𝑀||𝜃_𝑛−𝜃_𝑛−1||+||𝜆_𝑛−𝜆_𝑛−1||.(3.36) Apply Lemma 2.1 to (3.36) to conclude that ‖𝑥_𝑛+1−𝑥_𝑛‖→0 as 𝑛→∞.
(3) We prove that 𝜔_𝑤(𝑥_𝑛)⊂𝑆. Let 𝑥̂∈𝜔_𝑤(𝑥_𝑛), and assume that 𝑥_𝑛𝑗⇀𝑥̂ for some subsequence {𝑥_𝑛𝑗}^∞𝑗=1 of {𝑥_𝑛}^∞𝑛=0. We may further assume that 𝜆_𝑛𝑗→𝜆∈[0,2/𝐿] due to condition (1.4). Set 𝑉∶=Proj_𝐶(𝐼−𝜆∇𝑓). Notice that 𝑉 is nonexpansive and Fix𝑉=𝑆. It turns out that ‖‖𝑥_𝑛𝑗−𝑉𝑥_𝑛𝑗‖‖≤≤‖‖𝑥‖‖𝑥_{𝑛𝑗𝑛𝑗}−𝑉_𝑛𝑗𝑥_𝑛𝑗‖‖+‖‖𝑉_𝑛𝑗𝑥_𝑛𝑗−𝑉𝑥_𝑛𝑗‖‖≤‖‖𝑥_𝑛𝑗−𝑥_𝑛𝑗+1‖‖+‖‖𝑥_𝑛𝑗+1−𝑉_𝑛𝑗𝑥_𝑛𝑗‖‖+‖‖𝑉_𝑛𝑗𝑥_𝑛𝑗−𝑉𝑥_𝑛𝑗‖‖+‖‖Pro−j_𝐶𝑥_𝑛𝑗+1‖‖+𝜃_𝑛𝑗‖‖𝛾ℎ𝑥_𝑛𝑗−𝜇𝐹𝑉^𝑛_𝑗𝑥_𝑛𝑗‖‖𝐼−𝜆^𝑛_𝑗∇𝑓𝑥_𝑛𝑗−Proj_𝐶(𝐼−𝜆∇𝑓)𝑥_𝑛𝑗≤‖‖𝑥_𝑛𝑗‖‖−𝑥_𝑛𝑗+1‖‖+𝜃_𝑛𝑗‖‖𝛾ℎ𝑥_𝑛𝑗−𝜇𝐹𝑉^𝑛_𝑗𝑥_𝑛𝑗‖‖+|||𝜆−𝜆_𝑛𝑗|||‖‖∇𝑓𝑥_𝑛𝑗≤‖‖𝑥_𝑛𝑗‖‖−𝑥_𝑛𝑗+1‖‖+2𝑀𝜃_𝑛𝑗+|||𝜆−𝜆_𝑛𝑗|||⟶0as𝑗⟶∞.(3.37) So Lemma 2.2 guarantees that 𝜔_𝑤(𝑥_𝑛)⊂Fix𝑉=𝑆.
(4) We prove that 𝑥_𝑛→𝑥^∗ as 𝑛→∞, where 𝑥^∗ is the unique solution of the 𝑉𝐼 (3.12). First observe that there is some 𝑥̂∈𝜔_𝑤(𝑥_𝑛)⊂𝑆 Such that limsup_𝑛→∞⟨(𝜇𝐹−𝛾ℎ)𝑥^∗,𝑥_𝑛−𝑥^∗⟩=⟨(𝜇𝐹−𝛾ℎ)𝑥^∗,𝑥̂−𝑥^∗⟩≥0.(3.38)
We now compute ‖‖𝑥_𝑛+1−𝑥^∗‖‖²=‖‖𝜃_𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜇𝜃_𝑛𝐹Proj_𝐶𝐼−𝜆_𝑛∇𝑓𝑥_𝑛−𝑥^∗‖‖²=‖‖𝜃_𝑛𝛾ℎ𝑥_𝑛−𝜇𝐹𝑥^∗+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥^∗‖‖²=‖‖𝜃_𝑛𝛾ℎ𝑥_𝑛−ℎ𝑥^∗+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−(𝐼−𝜇𝜃_𝑛𝐹)𝑉_𝑛𝑥^∗+𝜃_𝑛𝛾ℎ𝑥^∗−𝜇𝐹𝑥^∗‖‖²≤‖‖𝜃_𝑛𝛾ℎ𝑥_𝑛−ℎ𝑥^∗+𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥^∗‖‖²=+‖‖𝜃2_𝑛𝜃_𝑛(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗𝛾ℎ𝑥_𝑛−ℎ𝑥^∗‖‖²+‖‖𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥^∗‖‖²++22𝜃𝜃_𝑛𝑛𝛾⟨ℎ𝑥_𝑛−ℎ𝑥^∗,𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥_𝑛−𝐼−𝜇𝜃_𝑛𝐹𝑉_𝑛𝑥^∗⟩≤𝜃_2𝑛⟨(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗⟩𝛾²𝜌²‖‖𝑥_𝑛−𝑥^∗‖‖²+1−𝜃_𝑛𝜏²‖‖𝑥_𝑛−𝑥^∗‖‖²+2𝜃_𝑛𝛾𝜌1−𝜃_𝑛𝜏‖‖𝑥_𝑛−𝑥^∗‖‖²=+𝜃2_2𝑛𝜃_𝑛(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗𝛾²𝜌²+1−𝜃_𝑛𝜏²+2𝜃_𝑛𝛾𝜌1−𝜃_𝑛𝜏‖‖𝑥_𝑛−𝑥^∗‖‖²≤+𝜃2_𝑛𝜃_𝑛(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗𝛾²𝜌²+1−2𝜃_𝑛𝜏+𝜃_𝑛𝜏²+2𝜃_𝑛𝛾𝜌‖‖𝑥_𝑛−𝑥^∗‖‖²=+12𝜃−_𝑛𝜃_𝑛(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗2𝜏−𝛾²𝜌²−𝜏²−2𝛾𝜌‖‖𝑥_𝑛−𝑥^∗‖‖²+2𝜃_𝑛(𝛾ℎ−𝜇𝐹)𝑥^∗,𝑥_𝑛+1−𝑥^∗.(3.39) Applying Lemma 2.1 to the inequality (3.39), together with (3.38), we get ‖𝑥_𝑛−𝑥^∗‖→0 as 𝑛→∞.

Corollary 3.6 (see [11]). Let {𝑥_𝑛} be generated by the following algorithm: 𝑥_𝑛+1=𝜃_𝑛ℎ𝑥_𝑛+1−𝜃_𝑛Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,∀𝑛≥0.(3.40) Assume that the sequence {𝜆_𝑛}^∞𝑛=0 satisfies the conditions (1.4) and (iv) and that {𝜃_𝑛}⊂[0,1] satisfies the conditions (i)–(iii). Then {𝑥_𝑛} converges in norm to 𝑥^∗ obtained in Corollary 3.4.

Corollary 3.7. Let {𝑥_𝑛} be generated by the following algorithm: 𝑥_𝑛+1=𝜃_𝑛𝛾ℎ𝑥_𝑛+𝐼−𝜃_𝑛𝐴Proj_𝐶𝑥_𝑛−𝜆_𝑛∇𝑓𝑥_𝑛,∀𝑛≥0.(3.41) Assume that the sequences {𝜃_𝑛} and {𝜆_𝑛} satisfy the conditions contained in Theorem 3.5, then {𝑥_𝑛} converges in norm to 𝑥^∗ obtained in Corollary 3.3.

Acknowledgments

Ming Tian is Supported in part by The Fundamental Research Funds for the Central Universities (the Special Fund of Science in Civil Aviation University of China: No. ZXH2012 K001) and by the Science Research Foundation of Civil Aviation University of China (No. 2012KYM03).