A Penalization-Gradient Algorithm for Variational Inequalities

Abstract

This paper is concerned with the study of a penalization-gradient algorithm for solving variational inequalities, namely, find 𝑥∈𝐶 such that ⟨𝐴𝑥,𝑦−𝑥⟩≥0 for all 𝑦∈𝐶, where 𝐴∶𝐻→𝐻 is a single-valued operator, 𝐶 is a closed convex set of a real Hilbert space 𝐻. Given Ψ∶𝐻→ℝ∪{+∞} which acts as a penalization function with respect to the constraint 𝑥∈𝐶, and a penalization parameter 𝛽_𝑘, we consider an algorithm which alternates a proximal step with respect to 𝜕Ψ and a gradient step with respect to 𝐴 and reads as 𝑥_𝑘=(𝐼+𝜆_𝑘𝛽_𝑘𝜕Ψ)⁻¹(𝑥_𝑘−1−𝜆_𝑘𝐴𝑥_𝑘−1). Under mild hypotheses, we obtain weak convergence for an inverse strongly monotone operator and strong convergence for a Lipschitz continuous and strongly monotone operator. Applications to hierarchical minimization and fixed-point problems are also given and the multivalued case is reached by replacing the multivalued operator by its Yosida approximate which is always Lipschitz continuous.

1. Introduction

Let 𝐻 be a real Hilbert space, 𝐴∶𝐻→𝐻 a monotone operator, and let 𝐶 be a closed convex set in 𝐻, we are interested in the study of a gradient-penalization algorithm for solving the problem of finding 𝑥∈𝐶 such that 𝐴𝑥,𝑦−𝑥≥0∀𝑦∈𝐶, (1.1) or equivalently 𝐴𝑥+𝑁_𝐶(𝑥)∋0, (1.2) where 𝑁_𝐶 is the normal cone to a closed convex set 𝐶. The above problem is a variational inequality, initiated by Stampacchia [1], and this field is now a well-known branch of pure and applied mathematics, and many important problems can be cast in this framework.

In [2], Attouch et al., based on seminal work by Passty [3], solve this problem with a multivalued operator by using splitting proximal methods. A drawback is the fact that the convergence in general is only ergodic. Motivated by [2, 4] and by [5] where penalty methods for variational inequalities with single-valued monotone maps are given, we will prove that our proposed forward-backward penalization-gradient method (1.9) enjoys good asymptotic convergence properties. We will provide some applications to hierarchical fixed-point and optimization problems and also propose an idea to reach monotone variational inclusions.

To begin with, see, for instance [6], let us recall that an operator with domain 𝐷(𝑇) and range 𝑅(𝑇) is said to be monotone if 𝑢−𝑣,𝑥−𝑦≥0whenever𝑢∈𝑇(𝑥),𝑣∈𝑇𝑦. (1.3) It is said to be maximal monotone if, in addition, its graph, gph𝑇∶={(𝑥,𝑦)∈𝐻×𝐻∶𝑦∈𝑇(𝑥)}, is not properly contained in the graph of any other monotone operator. An operator sequence 𝑇_𝑘 is said to be graph convergent to 𝑇 if (gph(𝑇_𝑘)) converges to gph(𝑇) in the Kuratowski-Painlevé's sense, that is, limsup_𝑘gph(𝑇_𝑘)⊂gph(𝑇)⊂liminf_𝑘gph(𝑇_𝑘). It is well-known that for each 𝑥∈𝐻 and 𝜆>0 there is a unique 𝑧∈𝐻 such that 𝑥∈(𝐼+𝜆𝑇)𝑧. The single-valued operator 𝐽^𝑇_𝜆∶=(𝐼+𝜆𝑇)⁻¹ is called the resolvent of 𝑇 of parameter 𝜆. It is a nonexpansive mapping which is everywhere defined and is related to its Yosida approximate, namely 𝑇_𝜆(𝑥)∶=(𝑥−𝐽^𝑇_𝜆(𝑥))/𝜆, by the relation 𝑇_𝜆(𝑥)∈𝑇(𝐽^𝑇_𝜆(𝑥)). The latter is 1/𝜆-Lipschitz continuous and satisfies (𝑇_𝜆)_𝜇=𝑇_𝜆+𝜇. Recall that the inverse 𝑇⁻¹ of 𝑇 is the operator defined by 𝑥∈𝑇⁻¹(𝑦)⇔𝑦∈𝑇(𝑥) and that, for all 𝑥,𝑦∈𝐻, we have the following key inequality 𝐽^𝑇_𝜆(𝑥)−𝐽^𝑇_𝜆𝑦²≤𝑥−𝑦²+𝐼−𝐽^𝑇_𝜆(𝑥)−𝐼−𝐽^𝑇_𝜆𝑦². (1.4) Observe that the relation (𝑇_𝜆)_𝜇(𝑥)=𝑇_𝜆+𝜇(𝑥) leads to 𝐽^𝑇𝜆_𝜇(𝑥)=𝜆𝜆𝜆+𝜇𝑥+1−𝐽𝜆+𝜇^{𝑇𝜆+𝜇}(𝑥). (1.5) Now, given a proper lower semicontinuous convex function 𝑓∶𝐻→ℝ∪{+∞}, the subdifferential of 𝑓 at 𝑥 is the set 𝜕𝑓(𝑥)=𝑢∈𝐻∶𝑓𝑦≥𝑓(𝑥)+𝑢,𝑦−𝑥∀𝑦∈𝐻. (1.6) Its Moreau-Yosida approximate and proximal mapping 𝑓_𝜆 and prox_𝜆𝑓 are given, respectively, by 𝑓_𝜆(𝑥)=inf_𝑦∈𝐻𝑓𝑦+12𝜆𝑦−𝑥²,prox_𝜆𝑓(𝑥)=argmin_𝑦∈𝐻𝑓𝑦+12𝜆𝑦−𝑥². (1.7) We have the following interesting relation (𝜕𝑓)_𝜆=∇𝑓_𝜆. Finally, given a nonempty closed convex set 𝐶⊂𝐻, its indicator function is defined as 𝛿_𝐶(𝑥)=0 if 𝑥∈𝐶 and +∞ otherwise. The projection onto 𝐶 at a point 𝑢 is 𝑃_𝐶(𝑢)=inf_𝑐∈𝐶‖𝑢−𝑐‖. The normal cone to 𝐶 at 𝑥 is 𝑁_𝐶(𝑥)={𝑢∈𝐻∶⟨𝑢,𝑐−𝑥⟩≤0∀𝑐∈𝐶} (1.8) if 𝑥∈𝐶 and ∅ otherwise. Observe that 𝜕𝛿_𝐶=𝑁_𝐶, prox_𝜆𝑓=𝐽_𝜆^𝜕𝑓, and 𝐽^𝑁𝐶_𝜆=𝑃_𝐶.

Given some 𝑥_𝑘−1∈𝐻, the current approximation to a solution of (1.2), we study the penalization-gradient iteration which will generate, for parameters 𝜆_𝑘>0,𝛽_𝑘→+∞, 𝑥_𝑘 as the solution of the regularized subproblem 1𝜆_𝑘𝑥_𝑘−𝑥_𝑘−1+𝐴𝑥_𝑘−1+𝛽_𝑘𝜕Ψ𝑥_𝑘∋0, (1.9) which can be rewritten as 𝑥_𝑘=𝐼+𝜆_𝑘𝛽_𝑘𝜕Ψ⁻¹𝑥_𝑘−1−𝜆_𝑘𝐴𝑥_𝑘−1. (1.10) Having in view a large range of applications, we shall not assume any particular structure or regularity on the penalization function Ψ. Instead, we just suppose that Ψ is convex, lower semicontinuous and 𝐶=argminΨ≠∅. We will denote by VI(𝐴,𝐶) the solution set of (1.2).

The following lemmas will be needed in our analysis, see for example [6, 7], respectively.

Lemma 1.1. Let 𝑇 be a maximal monotone operator, then (𝛽_𝑘𝑇) graph converges to 𝑁_𝑇⁻¹₍₀₎ as 𝛽_𝑘→+∞ provided that 𝑇⁻¹(0)≠∅.

Lemma 1.2. Assume that 𝛼_𝑘 and 𝛿_𝑘 are two sequences of nonnegative real numbers such that 𝛼_𝑘+1≤𝛼_𝑘+𝛿_𝑘. (1.11) If lim_𝑘→+∞𝛿_𝑘=0, then there exists a subsequence of (𝛼_𝑘) which converges. Furthermore, if ∑^∞_𝑘=0𝛿_𝑘<+∞, then lim_𝑘→+∞𝛼_𝑘 exists.

2. Main Results

2.1. Weak Convergence

Theorem 2.1. Assume that VI(𝐴,𝐶)≠∅, 𝐴 is inverse strongly monotone, namely ≥1𝐴𝑥−𝐴𝑦,𝑥−𝑦𝐿𝐴𝑥−𝐴𝑦²∀𝑥,𝑦∈𝐻,forsome𝐿>0. (2.1) If ^∞_𝑘=0𝑥−𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘𝑥−𝜆_𝑘𝐴𝑥<+∞∀𝑥∈VI(𝐴,𝐶), (2.2) and 𝜆_𝑘∈]𝜀,2/𝐿−𝜀[ (where 𝜀>0 is a small enough constant), then the sequence (𝑥_𝑘)_𝑘∈ℕ generated by algorithm (1.9) converges weakly to a solution of Problem (1.2).

Proof. Let 𝑥 be a solution of (1.2), observe that 𝑥 solves (1.2) if and only if 𝑥=(𝐼+𝜆_𝑘𝑁_𝐶)⁻¹(𝑥−𝜆_𝑘𝐴𝑥)=𝑃_𝐶(𝑥−𝜆_𝑘𝐴𝑥). Set 𝑥_𝑘=(𝐼+𝜆_𝑘𝛽_𝑘𝜕Ψ)⁻¹(𝑥−𝜆_𝑘𝐴𝑥), by the triangular inequality, we can write 𝑥_𝑘−𝑥≤𝑥_𝑘−𝑥_𝑘+𝑥_𝑘−𝑥. (2.3) On the other hand, by virtue of (1.4) and (2.1), we successively have 𝑥_𝑘−𝑥_𝑘²≤𝑥_𝑘−1−𝑥−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥²−𝑥_𝑘−1−𝑥_𝑘−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥+𝑥_𝑘−𝑥²≤𝑥_𝑘−1−𝑥²−𝜆_𝑘2𝐿−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥²−𝑥_𝑘−1−𝑥_𝑘−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥+𝑥_𝑘−𝑥². (2.4) Hence 𝑥_𝑘−𝑥<𝑥_𝑘−1−𝑥²−𝜀²𝐴𝑥_𝑘−1−𝐴𝑥²−𝑥_𝑘−1−𝑥_𝑘−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥+𝑥_𝑘−𝑥²+𝑥−𝑥_𝑘. (2.5)
The later implies, by Lemma 1.2 and the fact that (2.2) insures lim_𝑘→+∞‖𝑥−𝑥_𝑘‖=0, that the positive real sequence (‖𝑥_𝑘−𝑥‖²)_𝑘∈ℕ converges to some limit 𝑙(𝑥), that is, 𝑙(𝑥)=lim_𝑘→+∞𝑥_𝑘−𝑥²<+∞, (2.6) and also assures that lim_𝑘→+∞𝐴𝑥_𝑘−1−𝐴𝑥²=0,lim_𝑘→+∞𝑥_𝑘−1−𝑥_𝑘−𝜆_𝑘𝐴𝑥_𝑘−1−𝐴𝑥+𝑥_𝑘−𝑥²=0. (2.7) Combining the two latter equalities, we infer that lim_𝑘→+∞𝑥_𝑘−1−𝑥_𝑘²=0. (2.8) Now, (1.9) can be written equivalently as 𝑥_𝑘−1−𝑥_𝑘𝜆_𝑘+𝐴𝑥_𝑘−𝐴𝑥_𝑘−1∈𝐴+𝛽_𝑘𝜕Ψ𝑥_𝑘. (2.9) By virtue of Lemma 1.1, we have (𝛽_𝑘𝜕Ψ) graph converges to 𝑁_argminΨ because (𝜕Ψ)⁻¹(0)=𝜕Ψ^∗(0)=argminΨ. (2.10) Furthermore, the Lipschitz continuity of 𝐴 (see, e.g., [8]) clearly ensures that the sequence (𝐴+𝛽_𝑘𝜕Ψ)graph converges in turn to 𝐴+𝑁_argminΨ.
Now, let 𝑥^∗ be a cluster point of {𝑥_𝑘}. Passing to the limit in (2.9), on a subsequence still denoted by {𝑥_𝑘}, and taking into account the fact that the graph of a maximal monotone operator is weakly strongly closed in 𝐻×𝐻, we then conclude that 0∈𝐴+𝑁_𝐶𝑥^∗, (2.11) because 𝐴 is Lipschitz continuous, (𝑥_𝑘) is asymptotically regular thanks to (2.8), and (𝜆_𝑘) is bounded away from zero.
It remains to prove that there is no more than one cluster point, our argument is classical and is presented here for completeness.
Let 𝑥 be another cluster of {𝑥_𝑘}, we will show that 𝑥=𝑥^∗. This is a consequence of (2.6). Indeed, 𝑙𝑥^∗=lim_𝑘→+∞𝑥_𝑘−𝑥^∗2,𝑙(𝑥)=lim_𝑘→+∞𝑥_𝑘−𝑥², (2.12) from 𝑥_𝑘−𝑥²=𝑥_𝑘−𝑥^∗2+𝑥^∗−𝑥²+2𝑥_𝑘−𝑥^∗,𝑥^∗−𝑥, (2.13) we see that the limit of ⟨𝑥_𝑘−𝑥^∗,𝑥^∗−𝑥⟩ as 𝑘→+∞ must exists. This limit has to be zero because 𝑥^∗ is a cluster point of {𝑥_𝑘}. Hence at the limit, we obtain 𝑙(𝑥)=𝑙𝑥^∗+𝑥^∗−𝑥². (2.14) Reversing the role of 𝑥 and 𝑥^∗, we also have 𝑙𝑥^∗=𝑙(𝑥)+𝑥^∗−𝑥². (2.15) That is 𝑥=𝑥^∗, which completes the proof.

Remark 2.2. (i) Note that, we can remove condition (2.2), but in this case we obtain that there exists a subsequence of (𝑥_𝑘) such that every weak cluster point is a solution of problem (1.2). This follows by Lemma 1.2 combined with the fact that 𝑥=𝐽^𝜕𝛿𝐶_𝜆∗(𝑥−𝜆^∗𝐴𝑥) and that (𝛽_𝑘𝜕Ψ) graph converges to 𝜕𝛿_𝐶. The later is equivalent, see for example [6], to the pointwise convergence of 𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘 to 𝐽^𝜕𝛿𝐶_𝜆∗ and therefore ensures that lim_𝑘→+∞𝑥−𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘𝑥−𝜆_𝑘𝐴𝑥=0. (2.16)
(ii) In the special case Ψ(𝑥)=(1/2)dist(𝑥,𝐶)², (2.2) reduces to ∑^∞_𝑘=01/𝛽_𝑘<+∞, see Application (2) of Section 3.
Suppose now that Ψ(𝑥)=dist(𝑥,𝐶), it well-known that prox_𝛾Ψ(𝑥)=𝑃_𝐶(𝑥) if dist(𝑥,𝐶)≤𝛾. Consequently, 𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘(𝑥)=𝑃_𝐶(𝑥)ifdist(𝑥,𝐶)≤𝜆_𝑘𝛽_𝑘, (2.17) which is the case for all 𝑘≥𝜅 for some 𝜅∈ℕ because (𝜆_𝑘) is bounded and lim_𝑘→+∞𝛽_𝑘=+∞. Hence lim_𝑘→+∞‖𝑥−𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘(𝑥−𝜆_𝑘𝐴𝑥)‖=0, for all 𝑘≥𝜅, and thus (2.2) is clearly satisfied.
The particular case Ψ=0 corresponds to the unconstrained case, namely, 𝐶=𝐻. In this context the resolvent associated to 𝛽_𝑘𝜕Ψ is the identity, and condition (2.2) is trivially satisfied.

2.2. Strong Convergence

Now, we would like to stress that we can guarantee strong convergence by reinforcing assumptions on 𝐴.

Proposition 2.3. Assume that 𝐴 is strong monotone with constant 𝛼>0, that is, 𝐴𝑥−𝐴𝑦,𝑥−𝑦≥𝛼𝑥−𝑦²∀𝑥,𝑦∈𝐻,forsome𝛼>0, (2.18) and Lipschitz continuous with constant 𝐿>0, that is, 𝐴𝑥−𝐴𝑦≤𝐿𝑥−𝑦∀𝑥,𝑦∈𝐻,forsome𝐿>0. (2.19) If 𝜆_𝑘∈]𝜀,2𝛼/𝐿²−𝜀[ (where 𝜀>0 is a small enough constant) and lim_𝑘→+∞𝜆_𝑘=𝜆^∗>0, then the sequence generated by (1.9) strongly converges to the unique solution of (1.2).

Proof. Indeed, by replacing inverse strong monotonicity of 𝐴 by strong monotonicity and Lipschitz continuity, it is easy to see from the first part of the proof of Theorem 2.1 that the operator of 𝐼−𝜆_𝑘𝐴 satisfies 𝐼−𝜆_𝑘𝐴(𝑥)−𝐼−𝜆_𝑘𝐴𝑦²≤1−2𝜆_𝑘𝛼+𝜆²_𝑘𝐿²𝑥−𝑦². (2.20) Following the arguments in the proof of Theorem 2.1 to obtain 𝑥_𝑘−𝑥≤1−2𝜆_𝑘𝛼+𝜆²_𝑘𝐿²𝑥_𝑘−1−𝑥+𝛿_𝑘(𝑥)with𝛿_𝑘(𝑥)∶=𝑥−𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘𝑥−𝜆_𝑘𝐴𝑥. (2.21) Now, by setting Θ(𝜆)=1−2𝜆𝛼+𝜆²𝐿², we can check that 0<Θ(𝜆)<1 if and only if 𝜆_𝑘∈]0,2𝛼/𝐿²[, and a simple computation shows that 0<Θ(𝜆_𝑘)≤Θ^∗<1 with Θ^∗=max{Θ(𝜀),Θ(2𝛼/𝐿²−𝜀)}. Hence, 𝑥_𝑘−𝑥≤Θ^∗𝑘𝑥₀−𝑥+^𝑘−1_𝑗=0Θ^∗𝑗𝛿_𝑘−𝑗(𝑥). (2.22) The result follows from Ortega and Rheinboldt [9, page 338] and the fact that lim_𝑘→+∞𝛿_𝑘(𝑥)=0. The later follows thanks to the equivalence between graph convergence of the sequence of operators (𝛽_𝑘𝜕Ψ) to 𝜕𝛿_𝐶 and the pointwise convergence of their resolvent operators combined with the fact that lim_𝑘→+∞𝜆_𝑘=𝜆^∗.

3. Applications

(1) Hierarchical Convex Minimization Problems
Having in mind the connection between monotone operators and convex functions, we may consider the special case 𝐴=∇Φ, Φ being a proper lower semicontinuous differentiable convex function. Differentiability of Φ ensures that ∇Φ+𝑁_argminΨ=𝜕(Φ+𝛿_argminΨ) and (1.2) reads as min_{𝑥∈argminΨ}Φ(𝑥). (3.1) Using definition of the Moreau-Yosida approximate, algorithm (1.9) reads as 𝑥_𝑘=argmin_𝑦∈𝐻𝑓𝑦+12𝜆_𝑘𝑦−𝐼−𝜆_𝑘𝐴𝑥_𝑘−1². (3.2) In this case, it is well-known that the assumption (2.1) of inverse strong monotonicity of ∇Φ is equivalent to its 𝐿-Lipschitz continuity. If further we assume ∑^∞_𝑘=1𝛿_𝑘(𝑥)<+∞ for all 𝑥∈VI(∇Φ,𝐶) and 𝜆_𝑘∈]𝜀,2/𝐿−𝜀[, then by Theorem 2.1 we obtain weak convergence of algorithm (3.2) to a solution of (3.1). The strong convergence is obtained, thanks to Proposition 2.3, if in addition Ψ is strongly convex (i.e., there is 𝛼>0; 1−𝜇Ψ𝑥₁+𝜇Ψ𝑥₂≥Ψ1−𝜇𝑥₁+𝜇𝑥₂+𝛼2𝜇1−𝜇𝑥₁−𝑥₂² (3.3) for all 𝜇∈[0,1], all 𝑥₁,𝑥₂∈𝐻) and (𝜆_𝑘) a convergent sequence with 𝜆_𝑘∈]𝜀,2𝛼/𝐿²−𝜀[. Note that strong convexity of Ψ is equivalent to 𝛼-strong monotonicity of its gradient. A concrete example in signal recovery is the Projected Land weber problem, namely, min_𝑥∈𝐶Φ(𝑥)1∶=2‖𝐿𝑥−𝑧‖², (3.4)  𝐿 being a linear-bounded operator. Set 𝐴(𝑥)∶=∇Φ(𝑥)=𝐿^∗(𝐿𝑥−𝑧). Consequently, ∀𝑥,𝑦∈𝐻𝐴(𝑥)−𝐴𝑦=𝐿^∗𝐿𝑥−𝑦≤‖𝐿‖²𝑥−𝑦, (3.5) and 𝐴 is therefore Lipschitz continuous with constant ‖𝐿‖². Now, it is well-known that the problem possesses exactly one solution if 𝐿 is bounded below, that is, ∃𝜅>0∀𝑥∈𝐻‖𝐿(𝑥)‖≥𝜅‖𝑥‖. (3.6) In this case, 𝐴 is strongly monotone. Indeed, it is easily seen that 𝑓 is strongly convex: consider 𝑥,𝑦∈𝐻 and 𝜇∈]0,1[, one has 𝜇(𝐿𝑥−𝑧)+1−𝜇𝐿𝑦−𝑧²2≤𝜇‖𝐿𝑥−𝑧‖²2+1−𝜇𝐿𝑦−𝑧²2−𝜅²𝜇1−𝜇𝑥−𝑦²2. (3.7)

(2) Classical Penalization
In the special case where Ψ(𝑥)=(1/2)dist(𝑥,𝐶)², we have 𝜕Ψ(𝑥)=𝑥−Proj_𝐶(𝑥), (3.8) which is nothing but the classical penalization operator, see [10]. In this context, taking into account the fact that 𝜕𝑓_𝜆𝜇=∇𝑓_𝜆+𝜇,𝐽_𝜆^𝜕𝑓=𝐼−𝜆𝜕𝑓_𝜆=𝐼−𝜆∇𝑓_𝜆,𝛿_𝐶𝜆=1𝜆Ψ, (3.9) and that 𝑥 solves (1.2), and thus 𝑥=𝑃_𝐶(𝑥−𝜆_𝑘𝐴𝑥), we successively have 𝑥_𝑘−𝑥=𝐽^𝛽𝑘_𝜆^𝜕Ψ_𝑘𝑥−𝜆_𝑘𝐴𝑥−𝐽^𝑁𝐶_𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥=𝜆_𝑘𝛽_𝑘𝜕Ψ_𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥−𝑁_𝐶𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥=𝜆_𝑘𝛽_𝑘(𝜕Ψ)_{𝜆𝑘𝛽𝑘}𝑥−𝜆_𝑘𝐴𝑥−∇𝛿_𝐶𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥=𝜆_𝑘𝛽_𝑘𝜕𝛿_{𝐶1𝜆𝑘𝛽𝑘}𝑥−𝜆_𝑘𝐴𝑥−∇𝛿_𝐶𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥=𝜆_𝑘𝛽_𝑘∇𝛿_{𝐶1+𝜆𝑘𝛽𝑘}𝑥−𝜆_𝑘𝐴𝑥−∇𝛿_𝐶𝜆𝑘𝑥−𝜆_𝑘𝐴𝑥=𝜆_𝑘1𝜆_𝑘−𝛽_𝑘1+𝜆_𝑘𝛽_𝑘𝑥−𝜆_𝑘𝐴𝑥−𝑃_𝐶𝑥−𝜆_𝑘𝐴𝑥=11+𝜆_𝑘𝛽_𝑘𝜆_𝑘𝐴𝑥≤1𝛽_𝑘‖𝐴𝑥‖. (3.10) So condition on the parameters reduces to ∑^∞_𝑘=11/𝛽_𝑘<+∞, and algorithm (1.9) is nothing but a relaxed projection-gradient method. Indeed, using (1.5) and the fact that 𝐽^𝑁𝐶_𝜆=𝑃_𝐶, we obtain 𝑥_𝑘=11+𝜆_𝑘𝛽_𝑘𝜆𝐼+^𝑘𝛽_𝑘1+𝜆_𝑘𝛽_𝑘𝑃_𝐶𝐼−𝜆_𝑘𝐴𝑥_𝑘−1. (3.11) An inspection of the proof of Theorem 2.1 shows that the weak converges is assured with 𝜆_𝑘∈]𝜀,2/𝐿−𝜀[.

(3) A Hierarchical Fixed-Point Problem
Having in mind the connection between inverse strongly monotone operators and nonexpansive mappings, we may consider the following fixed-point problem: (𝐼−𝑃)𝑥+𝑁_𝐶(𝑥)∋0, (3.12) with 𝑃 a nonexpansive mapping, namely, ‖𝑃𝑥−𝑃𝑦‖≤‖𝑥−𝑦‖.
It is well-known that 𝐴=𝐼−𝑃 is inverse strongly monotone with 𝐿=2. Indeed, by definition of 𝑃, we have (𝐼−𝐴)𝑥−(𝐼−𝐴)𝑦≤𝑥−𝑦. (3.13) On the other hand (𝐼−𝐴)𝑥−(𝐼−𝐴)𝑦²=𝑥−𝑦²+𝐴𝑥−𝐴𝑦²−2𝑥−𝑦,𝐴𝑥−𝐴𝑦. (3.14) Combining the two last inequalities, we obtain ≥1𝑥−𝑦,𝐴𝑥−𝐴𝑦2𝐴𝑥−𝐴𝑦². (3.15) Therefore, by Theorem 2.1 we get the weak convergence of the sequence (𝑥_𝑘) generated by the following algorithm: 𝑥_𝑘=prox_𝛽𝑘Ψ𝐼−𝜆_𝑘𝑥_𝑘−1+𝜆_𝑘𝑃𝑥_𝑘−1 (3.16) to a solution of (3.12) provided that ∑^∞_𝑘=1𝛿_𝑘(𝑥)<+∞ for all 𝑥∈VI(𝐼−𝑃,𝐶) and 𝜆_𝑘∈]𝜀,1−𝜀[. The strong convergence of (1.9) is obtained, by applying Proposition 2.3, for 𝑃 a contraction mapping, namely, ‖𝑃𝑥−𝑃𝑦‖≤𝛾‖𝑥−𝑦‖ for 0<𝛾<1 which is equivalent to the (1−𝛾)-strong monotonicity of (𝐼−𝑃), and (𝜆_𝑘) is a convergent sequence with 𝜆_𝑘∈]𝜀,2(1−𝛾)/(1+𝛾)²−𝜀[. It is easily seen that in this case 𝐼−𝑃 is (1+𝛾)-Lipschitz continuous.

4. Towards the Multivalued Case

Now, we are interested in (1.2) when 𝐴∶𝐻→2^𝐻 is a multi-valued maximal monotone operator. With the help of the Yosida approximate which is always inverse strongly monotone (and thus single-valued), we consider the following partial regularized version of (1.2): 𝐴_𝛾𝑥^∗_𝛾+𝑁_𝐶𝑥^∗_𝛾∋0, (4.1) where 𝐴_𝛾 stands for the Yosida approximate of 𝐴.

It is well-known that 𝐴_𝛾 is inverse strongly monotone. More precisely, we have 𝐴_𝛾𝑥−𝐴_𝛾𝑦,𝑥−𝑦≥𝛾𝐴_𝛾𝑥−𝐴_𝛾𝑦². (4.2) Using definition of the Yosida approximate, algorithm (1.9) applied to (4.1) reads as 𝑥^𝛾_𝑘=𝐼+𝜆_𝑘𝛽_𝑘𝜕Ψ⁻¹𝜆1−^𝑘𝛾𝑥^𝛾_𝑘−1+𝜆_𝑘𝛾𝐽^𝐴_𝛾𝑥^𝛾_𝑘−1. (4.3) From Theorem 2.1, we infer that 𝑥^𝛾_𝑘 converges weakly to a solution 𝑥^𝛾 provided that 𝜆_𝑘∈]𝜀,2𝛾−𝜀[. Furthermore, it is worth mentioning that if 𝐴 is strongly monotone, 𝐴_𝛾 is also strongly monotone, and thus (4.1) has a unique solution 𝑥^𝛾. By a result in [8, page 35], we have the following estimate: 𝑥−𝑥^𝛾≤𝑜√𝛾. (4.4) Consequently, (4.3) provides approximate solutions to the variational inclusion (1.2) for small values of 𝛾. Furthermore, when 𝐴=∇Φ, we have (𝜕Φ)_𝛾(𝑥)+𝑁_𝐶(𝑥)=∇Φ_𝛾(𝑥)+𝑁_𝐶(𝑥)=𝜕Φ_𝛾+𝛿_𝐶(𝑥), (4.5) and thus (4.1) reduces to min_𝑥∈𝐶Φ_𝛾(𝑥). (4.6)

If (3.1) and (4.1) are solvable, by ([11] Theorem  3.3), we have for all 𝛾>0  0≤min_𝑥∈𝐶Φ(𝑥)−min_𝑥∈𝐶Φ_𝛾(𝑥)≤𝛾𝑦², (4.7) where 𝑦=∇Φ(𝑦)(∈−𝑁_𝐶(𝑥)) with 𝑥 a solution of (3.1). The value of (3.1) is thus close to those of (4.1) for small values of 𝛾, and hence, this confirmed the pertinence of the proposed approximation idea to reach the multi-valued case. Observe that in this context, algorithm (4.3) reads as 𝑥^𝛾_𝑘=prox_𝛽𝑘Ψ𝜆1−^𝑘𝛾𝑥^𝛾_𝑘−1+𝜆_𝑘𝛾prox_𝛾Φ𝑥^𝛾_𝑘−1. (4.8)

5. Conclusion

The authors have introduced a forward-backward penalization-gradient algorithm for solving variational inequalities and studied their asymptotic convergence properties. We have provided some applications to hierarchical fixed-point and optimization problems and also proposed an idea to reach monotone variational inclusions.

Acknowledgment

We gratefully acknowledge the constructive comments of the anonymous referees which helped them to improve the first version of this paper.