Convergence of a Proximal Point Algorithm for Solving Minimization Problems

Abstract

We introduce and consider a proximal point algorithm for solving minimization problems using the technique of Güler. This proximal point algorithm is obtained by substituting the usual quadratic proximal term by a class of convex nonquadratic distance-like functions. It can be seen as an extragradient iterative scheme. We prove the convergence rate of this new proximal point method under mild assumptions. Furthermore, it is shown that this estimate rate is better than the available ones.

1. Introduction

The purpose of this paper is twofold. Firstly, it proposes an extension of the proximal point method introduced by Güler [1] in 1992, where the usual quadratic proximal term is substituted by a class of strictly convex distance-like functions, called Bregman functions. Secondly, it offers a general framework for the convergence analysis of the proximal point method of Güler. This framework is general enough to apply different classes of Bregman functions and still yield simple convergence proofs. The methods being analyzable in this context are called Güler's generalized proximal point algorithm, and are closely related to the Bregman proximal methods [2–5]. The analysis, we develop is different from the works in [4, 5], since our method is based on Güler's technique.

2. Preliminaries

To be more specific, we consider the minimization problem in the following form:𝑓_∗∶=min𝑓(𝑥)∶𝑥∈ℝ^𝑛, (2.1) where 𝑓∶ℝ^𝑛→ℝ∪{∞} is a closed proper convex function. To solve the problem (2.1), Teboulle [6], Chen and Teboulle [2, 6], Eckstein [4] and Burachik [3] proposed a general scheme using the Bregman proximal mappings of the type 𝒥_𝑐𝒥_𝑐𝑦∶=argmin_{𝑥∈ℝ𝑛}𝑓(𝑥)+𝑐⁻¹𝐃_𝐡𝑥,𝑦, (2.2) where 𝐃_𝐡 is given by𝐃_𝐡𝑥,𝑦∶=ℎ(𝑥)−ℎ𝑦−∇ℎ𝑦,𝑥−𝑦, (2.3) with ℎ is a strictly convex and continuously differentiable function.

Throughout this paper, ‖⋅‖ denotes the 𝑙₂-norm and ⟨⋅,⋅⟩ denotes the Euclidean inner product in ℝ^𝑛. Let 𝐺 be a continuous single-valued mapping from ℝ^𝑛 into ℝ^𝑛. The mapping 𝐺 is Lipschitz continuous with Lipschitz constant 𝐿, if for all 𝑥,𝑦∈ℝ^𝑛,‖𝐺(𝑥)−𝐺(𝑦)‖≤𝐿‖𝑥−𝑦‖. We denote also by 𝜌(𝑥,𝑋) the distance of 𝑥 to the set 𝑋 and it is given by 𝜌(𝑥,𝑋)=min_𝑦∈𝑋‖𝑥−𝑦‖. Further notations and definitions used in this paper are standard in convex analysis and may be found in Rockafellar's book [7].

This type of kernels was introduced first by [8] in 1967. The corresponding algorithm using these Bregman proximal mappings is called the Generalized Proximal Point Method (GPPM) and known also under the terminology of Bregman Proximal Methods. These proximal method solve (2.1) by considering a sequence of unconstrained minimization problems, which can be summed as follows.

Algorithm 2.1. (1) Initialize 𝑥⁰∈ℝ^𝑛∶𝑓(𝑥⁰)<∞,𝑐₀>0.
(2) Compute the solution 𝑥^𝑘+1 by the iterative scheme: 𝑥^𝑘+1∶=argmin_{𝑥∈ℝ𝑛}𝑓(𝑥)+𝑐_𝑘⁻¹𝐃_𝐡𝑥,𝑥^𝑘, (2.4) where {𝑐_𝑘} is a sequence of positive numbers and 𝐃_𝐡(⋅,⋅) is defined by (2.3).

For 𝐃_𝐡(𝑥,𝑦)=(1/2)‖𝑥−𝑦‖², Algorithm 2.1 coincides with the classical proximal point algorithm (PPA) introduced by Moreau [9] and Martinet [10].

Under mild assumptions on the data of (2.1) ergodic convergence was proved [2, 5] when 𝜎_𝑛=∑^𝑛_𝑘=1𝑐_𝑘→∞ with the following global rate of convergence estimate:𝑓𝑥^𝑛(𝑥)1−𝑓=𝒪𝜎_𝑛,∀𝑥∈ℝ^𝑛. (2.5) Our purpose in this paper is to propose an algorithm of the same type as Algorithm 2.1 which has better convergence rate. To this goal, we propose to combine Güler's scheme [1] and the Bregman proximal method. The main difference concerns the generation of an additional sequence {𝑦_𝑘}⊂ℝ^𝑛 in the unconstrained minimization (2.4) in such a way: 𝑥^𝑘+1∶=argmin_{𝑥∈ℝ𝑛}𝑓(𝑥)+𝑐_𝑘⁻¹𝐃_𝐡𝑥,𝑦_𝑘. (2.6) We show (see Section 4) that this new proximal method possesses the following rate estimate𝑓𝑥^𝑛(𝑥)1−𝑓=𝒪𝛾²_𝑛,∀𝑥,𝛾_𝑛=^𝑛_𝑘=1√𝑐_𝑘, (2.7) which is faster than (2.5). Further, the convergence in terms of the objective values occurs when 𝛾_𝑛→∞ which is weaker than 𝜎_𝑛→∞.

We briefly recall here the notion of Bregman functions called also 𝐃-functions introduced by Brègman ([8], 1967), developed and used in the proximal theory by [4, 6, 11–13]. Let 𝑆 be an open subset of ℝ^𝑛 and let ℎ∶𝑆→ℝ be a finite-valued continuously differentiable function on 𝑆 be and let 𝐃_𝐡 defined by𝐃_𝐡𝑥,𝑦∶=ℎ(𝑥)−ℎ𝑦−∇ℎ𝑦,𝑥−𝑦,𝑥∈𝑆,𝑦∈𝑆. (2.8)

Definition 2.2. ℎ is called a Bregman function with zone 𝑆 or a 𝐷-function if:(a)ℎ is continuously differentiable on 𝑆 and continuous on 𝑆,(b)ℎ is strictly convex on 𝑆,(c)for every 𝜆∈ℝ, the partial level sets 𝐿₁(𝑦,𝜆)={𝑥∈𝑆∶𝐃_𝐡(𝑥,𝑦)≤𝜆} and 𝐿₂(𝑥,𝜆)={𝑦∈𝑆∶𝐃_𝐡(𝑥,𝑦)≤𝜆} are bounded for every 𝑦∈𝑆 and 𝑥∈𝑆, respectively,(d)if {𝑦_𝑘}∈𝑆 is a convergent sequence with limit 𝑦^∗, then 𝐃_𝐡(𝑦^∗,𝑦_𝑘)→0,(e)if {𝑥^𝑘} and {𝑦_𝑘} are sequences such that 𝑦_𝑘→𝑦^∗∈𝑆, {𝑥^𝑘} is bounded and 𝐃_𝐡(𝑥^𝑘,𝑦_𝑘)→0, then 𝑥^𝑘→𝑦^∗.

From the above definition, we extract the following properties (see, for instance, [6, 13]).

Lemma 2.3. Let ℎ be a Bregman function with zone 𝑆. Then,(i)𝐃_𝐡(𝑥,𝑥)=0 and 𝐃_𝐡(𝑥,𝑦)≥0 for 𝑥∈𝑆 and 𝑦∈𝑆,(ii)for all 𝑎, 𝑏∈𝑆 and 𝑐∈𝑆, 𝐃_𝐡(𝑐,𝑎)+𝐃_𝐡(𝑎,𝑏)−𝐃_𝐡(𝑐,𝑏)=⟨∇ℎ(𝑏)−∇ℎ(𝑎),𝑐−𝑎⟩, (2.9)(iii)for all 𝑎,𝑏∈𝑆, 𝐃_𝐡(𝑎,𝑏)+𝐃_𝐡(𝑏,𝑎)≤‖∇ℎ(𝑎)−∇ℎ(𝑏)‖×‖𝑎−𝑏‖. (2.10)(iv)for all 𝑥∈𝑆ℎ^∗(∇ℎ(𝑥))=⟨𝑥,∇ℎ(𝑥)⟩−ℎ(𝑥),(v)let {𝑥^𝑘}∈𝑆 such that 𝑥^𝑘→𝑥^∗∈𝑆, then 𝐃_𝐡(𝑥^∗,𝑥^𝑘)→0 and 𝐃_𝐡(𝑥^𝑘,𝑥^∗)→0.

Lemma 2.4. (i) Let 𝑔∶ℝ^𝑛→ℝ be a strictly convex function such that 𝑔∈𝐶²ℝ^𝑛,lim_{‖𝑥‖→∞}𝑔(𝑥)‖𝑥‖=+∞, (2.11) then 𝑔 is a Bregman function.
(ii) If 𝑔 is a Bregman function, then 𝑔(𝑥)+𝑐^⊤𝑥+𝑑 for any 𝑐∈ℝ^𝑛,𝑑∈ℝ, also is a Bregman function.

Remark 2.5. 𝐃_𝐡(⋅,⋅) cannot be considered as a distance because of the lack of the triangle inequality and the symmetry property. 𝐃_𝐡(⋅,⋅) is usually called an entropy distance.

The paper is organized as follows. In Section 3, we recall briefly the proximal point method of Güler. Section 4 will be devoted to the presentation and convergence analysis of the proposed algorithm. Finite convergence is shown in Section 5. Finally, in Section 6 we present an application of this method to solve variational inequalities problem.

3. Extragradient Algorithm

In 1992, Güler [1] has developed a new proximal point approach similar to the classical one (PPA) based on the idea stated by Nesterov [14].

Güler's proximal point algorithm (GPPA) can be summed up as follows.

Algorithm 3.1. (i) Initialize 𝑥⁰∈ℝ^𝑛∶𝑓(𝑥⁰)<∞,𝑐₀>0,𝐴>0.
Define 𝜈₀∶=𝑥⁰,𝐴₀∶=𝐴,𝑘=0.
(ii) Compute 𝛼_𝑘=((𝐴_𝑘𝑐_𝑘)²+4𝐴_𝑘𝑐_𝑘−𝐴_𝑘𝑐_𝑘)/2.
(iii) Compute the solution 𝑥^𝑘+1 by the iterative scheme: 𝑦_𝑘=1−𝛼_𝑘𝑥^𝑘+𝛼_𝑘𝜈_𝑘,𝑥^𝑘+1∶=argmin_{𝑧∈ℝ𝑛}𝑓(𝑧)+2𝑐_𝑘⁻¹𝑧−𝑦_𝑘²,𝜈_𝑘+1=𝜈_𝑘−𝑥^𝑘+1−𝑦_𝑘𝛼_𝑘,𝐴_𝑘+1=1−𝛼_𝑘𝐴_𝑘. (3.1)

For the convergence analysis, see Güler [1].

Remark 3.2. The GPPA can be seen as a suitable conjugate gradient type modification of the PPA of Rockafellar applied to (2.1).

4. Main Result

4.1. Introduction

The method that we are proposing is a modification of Güler's new proximal point approach GPPA discussed in Section 3 and can be considered as a nonlinear (or a nonquadratic) version of GPPA with Bregman kernels. In this paper it is shown that this method, which we call BGPPA possesses the strong convergence results obtained by Güler [1] and therefore this new scheme provides faster (global) convergence rates than the classical Bregman proximal point methods (cf. [2, 4–6, 11, 13, 15]). In this paper, we propose the following algorithm generalizing Güler's proximal point algorithm and summed up as follows.

Algorithm 4.1. (i) Initialize: 𝑥⁰∈ℝ^𝑛∶𝑓(𝑥⁰)<∞,𝑐₀>0,𝐴>0.
Define 𝜈₀∶=𝑥⁰,𝐴₀∶=𝐴,𝑘=0
(ii) Compute: 𝛼_𝑘 such that 𝛼²_𝑘=(1−𝛼_𝑘)𝐴_𝑘𝑐_𝑘/𝐿𝐿^∗.
(iii) Compute the solution 𝑥^𝑘+1 by the iterative scheme: 𝑦_𝑘=1−𝛼_𝑘𝑥^𝑘+𝛼_𝑘𝜈_𝑘,𝑥^𝑘+1∶=argmin_{𝑧∈ℝ𝑛}𝑓(𝑧)+𝑐_𝑘⁻¹𝐃_𝐡𝑧,𝑦_𝑘,𝜈_𝑘+1=∇ℎ^∗∇ℎ𝜈_𝑘+𝑥^𝑘+1−𝑦_𝑘𝛼_𝑘,𝐴_𝑘+1=1−𝛼_𝑘𝐴_𝑘. (4.1)

In this section we develop convergence results for the generalized Güler's proximal point algorithm GGPPA presented in Section 4.2. Our analysis is basically based on the following lemma.

Lemma 4.2 ([1, page 654]).   One has ^𝑘−1_𝑗=01−𝛼_𝑗≤11+√𝐴/2∑^𝑘−1_𝑗=0𝑐_𝑗² (4.2) for all 𝛼_𝑗∈[0,1[ and 𝐴>0.

Theorem 4.3. For all 𝑥∈𝒮 such that 𝑓(𝑥)<∞, one has the following convergence rate estimate: 𝑓𝑥^𝑘(𝑥)≤4𝑓𝑥−𝑓⁰−𝑓(𝑥)+𝐴𝐃_𝐡𝑥,𝑥⁰𝐴∑^𝑘−1_𝑗=0𝑐_𝑗². (4.3)

Proof. Using the fact that 𝜙₀(𝑥)∶=𝑓(𝑥⁰)+𝐴𝐃_𝐡(𝑥,𝑥⁰), 𝑥⁰∈𝒮, and Lemma 4.2, we obtain 𝑓𝑥^𝑘−𝑓(𝑥)≤^𝑘−1_𝑗=01−𝛼_𝑗𝜙₀(𝑥)(𝑥)≤1−𝑓1+√𝐴/2∑^𝑘−1_𝑗=0𝑐_𝑗²𝑓𝑥⁰+𝐴𝐃_𝐡𝑥,𝑥⁰−𝑓(𝑥). (4.4) Since [1+(√𝐴/2)∑^𝑘−1_𝑗=0𝑐_𝑗]²≥(∑^𝑘−1_𝑗=0𝑐_𝑗)²𝐴/4, then (4.3) holds.

Theorem 4.4. Consider the sequence {𝑥^𝑘} generated by GGPPA and let 𝑥^∗ be a minimizer of 𝑓(𝑥) on ℝ^𝑛. Assume that(1)ℎ is a Bregman function with zone 𝒮 such that Dom(𝜕𝑓)⊂𝒮,(2)Im(∇ℎ)=ℝ^𝑛 or Im(∇ℎ) is open,(3)∇ℎ is Lipschitz continuous with coefficient 𝐿,then(a)for all 𝑥⁰∈Dom(∇ℎ), the sequence {𝑥^𝑘} is well defined,(b) the GGPPA possesses this following convergence rate estimate: 𝑓𝑥^𝑘−𝑓_∗≤4𝑓𝑥⁰−𝑓_∗+𝜌𝑥⁰,𝑋^∗2(𝐴𝐿)/2𝐴∑^𝑘−1_𝑗=0𝑐_𝑗², (4.5)(c)𝑓(𝑥^𝑘)−𝑓_∗→0, when ∑^∞_𝑘=0√𝑐_𝑘=∞,(d)𝑓𝑥^𝑘−𝑓_∗1=𝒪𝑘² (4.6) if 𝑐_𝑘≥𝑐>0.

Proof. (a) Follows from [8, Theorem 4].
(b) Uses assumption (2.3) in the following manner 𝐃_𝐡𝑥,𝑥⁰=ℎ(𝑥)−ℎ𝑥⁰−∇ℎ𝑥⁰,𝑥−𝑥⁰≤¹₀∇ℎ𝑥⁰+𝑡𝑥−𝑥⁰−∇ℎ𝑥⁰𝑥−𝑥⁰𝑑𝑡≤𝐿𝑥−𝑥⁰²¹₀≤𝐿𝑡𝑑𝑡2𝑥−𝑥⁰², (4.7) and by taking 𝑥=𝑥^∗ in (4.3), then we have 𝑓𝑥^𝑘−𝑓_∗≤4𝑓𝑥⁰−𝑓_∗+(𝐴𝐿/2)𝑥^∗−𝑥⁰²𝐴∑^𝑘−1_𝑗=0𝑐_𝑗². (4.8) Since 𝑥^∗ is arbitrary, then (4.5) holds.
(c) Is obvious.
(d) It suffices to observe that if 𝑐_𝑘≥𝑐>0, we have ^𝑘−1_𝑗=0𝑐_𝑗²≥^𝑘−1_𝑗=0√𝑐²=𝑐𝑘². (4.9)

4.2. Finite Convergence

Note that the finite convergence property was established for the classical proximal point algorithm in the case of sharp minima, see, for example, [16]. Recently, Kiwiel [5] has extended this property to his generalized Bregman proximal method (BPM). In the following theorem we prove that Algorithm 3.1 has this property. Our proof is based on Kiwiel's one [5, Theorem 6.1 page 1151].

Definition 4.5. A closed proper convex function 𝑓∶ℝ^𝑛→ℝ is said to have a sharp minimum on ℝ^𝑛 if and only if there exists 𝜏>0 such that 𝑓(𝑥)≥min_ℝ𝑛𝑓+𝜏min_{𝑧∈Argmin𝑓}‖𝑥−𝑧‖∀𝑥. (4.10)

Theorem 4.6. Under the same hypothesis as in Theorem 4.4 and by considering GGPPA with 𝑓 having a sharp minimum on ℝ^𝑛 and 𝑐_𝑘 being bounded, then there exists 𝑘 such that 0∈𝜕𝑓(𝑥^𝑘) and 𝑥^𝑘∈𝑋^∗.

Proof. Straightforward, using Theorem 4.4 and [5, Theorem 6.1, page 1151].

5. Convergence Rate of GGPPA

If {𝑥^𝑘} is a sequence of points, one forms the sequence {𝑧^𝑛} of weighted averages given by𝑧^𝑛=^𝑛_𝑘=1𝑐_𝑘𝜎_𝑛𝑥^𝑘,𝜎_𝑛=^𝑛_𝑘=1𝑐_𝑘, (5.1) where 𝑐_𝑘>0. If the sequence {𝑧^𝑛} converges, then {𝑥^𝑘}_𝑘 is said to converge ergodically.

Theorem 5.1. GGPPA possesses the following convergence rate: 𝑓𝑥^𝑛−𝑓_∗1=𝑜𝜎_𝑛, (5.2) that is, 𝜎_𝑛(𝑓(𝑥^𝑛)−𝑓_∗)→0. Furthermore, if 𝑐_𝑘≥𝑐>0, then one has 𝑓𝑥^𝑛−𝑓_∗1=𝑜𝑛. (5.3)

Proof. Let 𝑥^∗ be a minimizer of 𝑓. For brevity, we denote 𝑊_𝑘=𝑓(𝑥^𝑘)−𝑓_∗,𝑉_𝑘=𝑓(𝑦^𝑘)−𝑓_∗ and Δ=∇ℎ(𝑦^𝑘)−∇ℎ(𝑥^𝑘+1). At optimality in the unconstrained minimization in GGPPA, we can write Δ𝑐_𝑘∈𝜕𝑓𝑥^𝑘+1, (5.4) and by the convexity of 𝑓, we have 𝑐_𝑘𝑓𝑥^𝑘+1−𝑓(𝑥)≤𝑥^𝑘+1−𝑥,Δ. (5.5) Setting 𝑥=𝑥^𝑘 in (5.5), we obtain 𝑐_𝑘𝑊_𝑘−𝑊_𝑘+1≥𝑥^𝑘−𝑥^𝑘+1,Δ, (5.6) and for 𝑥=𝑦^𝑘, we have 𝑐_𝑘𝑉_𝑘−𝑊_𝑘+1≥𝑦^𝑘−𝑥^𝑘+1,Δ. (5.7) Or again, if we set 𝑥=𝑥^∗ in (5.5), and using the Cauchy-Schwartz inequality, we obtain 𝑐_𝑘𝑓𝑥^𝑘+1−𝑓_∗≤𝑥^𝑘+1−𝑥^∗,Δ, (5.8) that is, 𝑐_𝑘𝑊_𝑘+1≤𝑥^𝑘+1−𝑦^𝑘,Δ+𝑦^𝑘−𝑥^∗,Δ. (5.9) Since ℎ is convex, ⟨𝑥^𝑘+1−𝑦^𝑘,Δ⟩≤0. Then we can write 𝑐_𝑘W_𝑘+1≤𝑦^𝑘−𝑥^∗,Δ, (5.10) that is, 𝑐²_𝑘𝑊²_𝑘+1≤𝑦^𝑘−𝑥^∗2‖Δ‖². (5.11) Using the relation ‖Δ‖²≤𝐿⟨Δ,𝑦^𝑘−𝑥^𝑘+1⟩ and the inequality (5.7), we get the relation 𝑐²_𝑘𝑊²_𝑘+1≤𝐿𝑐_𝑘𝑉_𝑘−𝑊_𝑘+1𝑦^𝑘−𝑥^∗2. (5.12) For short we denote 𝑀_𝑘=‖𝑦^𝑘−𝑥^∗‖ thus, (5.12) becomes 𝑐_𝑘𝑊²_𝑘+1+𝐿𝑀²_𝑘𝑊_𝑘+1≤𝐿𝑀²_𝑘𝑉_𝑘. (5.13) Then by dividing both terms by 𝐿𝑀²_𝑘>0, we get 𝑊_𝑘+1𝑐_𝑘𝐿𝑀²_𝑘𝑊_𝑘+1+1≤𝑉_𝑘. (5.14) Since the left-side term is positive, then 𝑉_𝑘⁻¹≤𝑊⁻¹_𝑘+1𝑐_𝑘𝐿𝑀²_𝑘𝑊_𝑘+1+1⁻¹. (5.15) Now following Güler [17, page 410], we use the fact that (1+𝑥)⁻¹≤1−2𝑥/3,forall𝑥∈[0,1/2]. To apply this inequality, it suffices to show that (𝑐_𝑘/𝐿𝑀²_𝑘)𝑊_𝑘+1 is less than or equal to 1/2. This can be deduced from this relation (see Lemma 2.3 (ii)): 𝑐_𝑘𝑊_𝑘+1≤Δ,𝑥^𝑘+1−𝑥^∗=𝐃_𝐡𝑥^∗,𝑦^𝑘−𝐃_𝐡𝑥^∗,𝑥^𝑘+1−𝐃_𝐡𝑥^𝑘+1,𝑦^𝑘. (5.16) Indeed, since 𝐃_𝐡(⋅,⋅)≥0, then (the proof of this next inequality can be found in the proof of Theorem 4.4-(b)) 𝑐_𝑘𝑊_𝑘+1≤𝐃_𝐡𝑥^∗,𝑦^𝑘≤𝐿2𝑥^∗−𝑦^𝑘2=𝐿𝑀²_𝑘2. (5.17) Therefore, 0<(𝑐_𝑘/𝐿𝑀²_𝑘)𝑊_𝑘+1≤1/2 and we obtain 0≤𝑉_𝑘⁻¹≤𝑊⁻¹_𝑘+11−2𝑐_𝑘3𝐿𝑀²_𝑘𝑊_𝑘+1. (5.18) To continue the proof, we will separate some different cases. Case 1.   If 𝑓(𝑥^𝑘+1)≤𝑓(𝑦^𝑘)≤𝑓(𝑥^𝑘). Then 𝑊_𝑘+1≤𝑉_𝑘≤𝑊_𝑘 and we have 𝑉_𝑘⁻¹≥𝑊_𝑘⁻¹. Thus, (5.18) becomes 𝑊_𝑘⁻¹≤𝑉_𝑘⁻¹≤𝑊⁻¹_𝑘+11−2𝑐_𝑘3𝐿𝑀²_𝑘𝑊_𝑘+1, (5.19) and by summation from 𝑘=0 to 𝑘=𝑛, we get 0≤^𝑛_𝑘=0𝑊_𝑘⁻¹≤^𝑛_𝑘=0𝑊⁻¹_𝑘+1−^𝑛_𝑘=02𝑐_𝑘3𝐿𝑀²_𝑘, (5.20) that is, 0≤𝑊₀⁻¹≤𝑊⁻¹_𝑛+1−^𝑛_𝑘=02𝑐_𝑘3𝐿𝑀²_𝑘. (5.21) Then, 0≤𝑊_𝑛+1≤3𝐿2∑^𝑛_𝑘=0𝑐_𝑘𝑦^𝑘−𝑥^∗−2. (5.22) Since 𝑥^∗ is an arbitrary solution, we can write 0≤𝑊_𝑛+1≤3𝐿2∑^𝑛_𝑘=0𝑐_𝑘𝜌𝑦^𝑘,𝑋^∗−2, (5.23) and by multiplying both terms by 𝜎_𝑛+1=∑^𝑛_𝑘=0𝑐_𝑘, we obtain 0≤𝜎_𝑛+1𝑊_𝑛+1≤3𝐿2∑^𝑛_𝑘=0𝑐_𝑘/𝜎_𝑛+1𝜌𝑦^𝑘,𝑋^∗−2. (5.24) Since 𝑦^𝑘 and 𝑥^𝑘+1 converge to the same point (indeed, we can see it via the formula giving 𝜈_𝑘+1 in the algorithm GGPPA and 𝜌(𝑥^𝑘+1,𝑋^∗)→0, then 𝜌(𝑥^𝑘+1,𝑋^∗)⁻²→∞; hence, we obtain 0≤𝜎_𝑛+1𝑓𝑥^𝑛+1−𝑓_∗⟶0, (5.25) which implies 𝑓𝑥^𝑛+1−𝑓_∗1=𝑜𝜎_𝑛+1, (5.26) that is, 𝑓𝑥^𝑛−𝑓_∗1=𝑜𝜎_𝑛. (5.27)Case 2.   If 𝑓(𝑥^𝑘+1)≤𝑓(𝑥^𝑘)≤𝑓(𝑦^𝑘). Then 𝑊_𝑘+1≤𝑊_𝑘≤𝑉_𝑘 and we have 𝑊⁻¹_𝑘+1≥𝑊_𝑘⁻¹ therefore; for 𝑛≥𝑘 we have 𝑊⁻¹_𝑛+1≥𝑊_𝑘⁻¹. Thus, using inequality (5.18), we write 𝑊⁻¹_𝑛+1≥𝑊⁻¹_𝑘+1≥2𝑐_𝑘3𝐿𝑀²_𝑘, (5.28) and by summation from 𝑘=0 to 𝑘=𝑛, we get (𝑛+1)𝑊⁻¹_𝑛+1≥^𝑛_𝑘=02𝑐_𝑘3𝐿𝑀²_𝑘. (5.29) Then, 0≤𝑊_𝑛+1≤3𝐿(𝑛+1)2∑^𝑛_𝑘=0𝑐_𝑘𝑦^𝑘−𝑥^∗−2. (5.30) Since 𝑥^∗ is an arbitrary solution, we can write 0≤𝑊_𝑛+1≤3𝐿(𝑛+1)2∑^𝑛_𝑘=0𝑐_𝑘𝜌𝑦^𝑘,𝑋^∗−2, (5.31) and by multiplying both terms by 𝜎_𝑛+1=∑^𝑛_𝑘=0𝑐_𝑘, we obtain 0≤𝜎_𝑛+1𝑊_𝑛+1≤3𝐿2∑^𝑛_𝑘=0𝑐_𝑘/(𝑛+1)𝜎_𝑛+1𝜌𝑦^𝑘,𝑋^∗−2. (5.32) Since 𝑦^𝑘 and 𝑥^𝑘+1 converge to the same point (indeed, we can see it via the formula giving 𝜈_𝑘+1 in the algorithm GGPPA and 𝜌(𝑥^𝑘+1,𝑋^∗)→0, then 𝜌(𝑥^𝑘+1,𝑋^∗)⁻²→∞; hence, we obtain 0≤𝜎_𝑛+1𝑓𝑥^𝑛+1−𝑓_∗⟶0, (5.33) which implies, 𝑓𝑥^𝑛+1−𝑓_∗1=𝑜𝜎_𝑛+1, (5.34) that is, 𝑓𝑥^𝑛−𝑓_∗1=𝑜𝜎_𝑛. (5.35)Case 3.   If 𝑓(𝑥^𝑘)≤𝑓(𝑥^𝑘+1)≤𝑓(𝑦^𝑘). In this case we observe that sequence {𝑓(𝑥^𝑘)}is increasing, which may imply a divergence of the approach.

Since 𝑓 is convex, then the following convergence rate estimate can be derived directly.

Corollary 5.2. If one assumes that 𝑐_𝑘≥𝑐>0 for all 𝑘, then 𝑓𝑧^𝑛−𝑓_∗1=𝑜𝑛. (5.36)

6. Conclusion

We have introduced an extragradient method to minimize convex problems. The algorithm is based on a generalization of the technique originally proposed by Nesterov [14] and readapted by Güler in [1, 17], where the usual quadratic proximal term was substituted by a class of convex nonquadratic distance-like functions. The new algorithm has a better theoretical convergence rate compared to the available ones. This motivates naturally the study of the numerical efficiency of the new algorithm and its application to solve variational inequality problems [18, 19]. Also, further efforts are needed to consider the given study for nonconvex situations and apply it to solve nonconvex equilibrium problems [20].

Acknowledgments

The authors would like to thank Dr. Osman Güler for providing them with reference [12] and the English translation of the original paper of Nesterov [14].