Convergence Analysis of the Relaxed Proximal Point Algorithm

Abstract

Recently, a worst-case 𝑂(1/𝑡) convergence rate was established for the Douglas-Rachford alternating direction method of multipliers (ADMM) in an ergodic sense. The relaxed proximal point algorithm (PPA) is a generalization of the original PPA which includes the Douglas-Rachford ADMM as a special case. In this paper, we provide a simple proof for the same convergence rate of the relaxed PPA in both ergodic and nonergodic senses.

1. Introduction

The finite-dimensional variational inequality (VI), denoted by VI(Ω,𝐹), is to find a vector 𝑤^∗∈Ω such that 𝑤−𝑤^∗𝑇𝐹𝑤^∗≥0,∀𝑤∈Ω, (1) where Ω is a nonempty closed convex set in ℜ^𝑛 and 𝐹 is a monotone mapping from ℜ^𝑛 into itself. The solution set, denoted by Ω^∗ is assumed to be nonempty. We refer to [1–4] for the pivotal roles of VIs in various fields such as economics, transportation, and engineering.

As is well known, proximal point algorithm (PPA), which was presented originally in [5] and mainly developed in [6, 7], is a well-developed approach to solving VI(Ω,𝐹). Let 𝑤^𝑘 be the current approximation of a solution of (1); then PPA generates the new iterate 𝑤^𝑘+1∈Ω by solving the following auxiliary VI: 𝑤−𝑤^𝑘+1𝑇𝐹𝑤^𝑘+1+1𝛽𝑤^𝑘+1−𝑤^𝑘≥0, (2) where 𝛽 is a positive constant. Compared to the monotone VI (1), (2) is easier to handle since it is a strongly monotone VI. In this paper, we focus on the relaxed proximal point algorithm (PPA) proposed by Gol’shtein and Tret’yakov in [8], which combines the PPA step (3a) with a relaxation step (3b) as follows:

𝑤^𝑘∈Ω,𝑤−𝑤^𝑘𝑇𝐹𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘𝑤≥0,∀𝑤∈Ω,_𝑘+1∶=𝑤^𝑘−𝛾𝑤^𝑘−𝑤^𝑘, (3a)(3b) where 𝛾∈(0,2) is a relaxation factor and 𝐺 is a symmetric positive semidefinite matrix. In particular, 𝛾 is called an under-relaxation factor when 𝛾∈(0,1) or an over-relaxation factor when 𝛾∈(1,2), and the relaxed PPA reduces to the original PPA (2) when 𝛾=1 and 𝐺=(1/𝛽)𝐼. For convenience, we still use the notation ‖𝑤‖²_𝐺 to represent the nonnegative number 𝑤^𝑇𝐺𝑤 in our analysis.

The Douglas-Rachford alternating direction methods of multipliers (ADMM) scheme proposed by Glowinski and Marrocco in [9] (see also [10]) is a commonplace tool to solve the convex minimization problem with linear constraints and a separable objective function as follows: min𝜃₁(𝑥)+𝜃₂𝑦∣𝐴𝑥+𝐵𝑦=𝑏,𝑥∈𝒳,𝑦∈𝒴, (4) where 𝐴∈ℜ^𝑚×𝑛1, 𝐵∈ℜ^𝑚×𝑛2, 𝑏∈ℜ^𝑚, 𝒳⊆ℜ^𝑛1, and 𝒴⊆ℜ^𝑛2 are closed convex sets and 𝜃₁: ℜ^𝑛1→ℜ and 𝜃₂: ℜ^𝑛2→ℜ are convex smooth functions. The iterative scheme of ADMM for solving (4) at the 𝑘-th iteration runs as 𝑥^𝑘+1∈𝒳,𝑥−𝑥^𝑘+1𝑇∇𝜃₁𝑥^𝑘+1−𝐴^𝑇𝜆^𝑘−𝐻𝐴𝑥^𝑘+1+𝐵𝑦^𝑘𝑦−𝑏≥0,∀𝑥∈𝒳,_𝑘+1∈𝒴,𝑦−𝑦^𝑘+1𝑇∇𝜃₂𝑦^𝑘+1−𝐵^𝑇𝜆^𝑘−𝐻𝐴𝑥^𝑘+1+𝐵𝑦^𝑘+1𝜆−𝑏≥0,∀𝑦∈𝒴,_𝑘+1∶=𝜆^𝑘−𝐻𝐴𝑥^𝑘+1+𝐵𝑦^𝑘+1−𝑏, (5a)(5b)(5c)

where 𝐻∶=ℎ𝐼 and ℎ is a positive constant. As shown in [11], ADMM can be regarded as an application of the relaxed PPA with 𝛾=1 (i.e., the original PPA (2)) and 𝐺=0000𝐵^𝑇𝐻𝐵−𝐵^𝑇0−𝐵𝐻⁻¹. (6) Without further assumption on 𝐵, the matrix 𝐺 defined previously can be guaranteed as a symmetric and positive semidefinite matrix. Recently, He and Yuan in [12] have shown a worst-case 𝑂(1/𝑡) convergence rate of the ADMM scheme (5a), (5b), and (5c) in an ergodic sense. You et al. in [13] have proved the same convergence rate of the Lagrangian PPA-based contraction methods with nonsymmetric linear proximal term in an ergodic sense. The purpose of this paper is to establish the 𝑂(1/𝑡) convergence rate of the relaxed PPA (3a) and (3b) in both ergodic and nonergodic senses.

2. Preliminaries

In this section, we review some preliminaries which are useful for further discussions. More specially, we recall a useful characterization on Ω^∗, the variational reformulation of (4), the relationship of the ADMM in [9, 10], and the relaxed PPA in [8] for solving this variational reformulation.

First, we provide a useful characterization on Ω^∗ as Theorem  2.3.5 in [14] and Theorem  2.1 in [12].

Theorem 1. The solution set of VI(Ω,𝐹) is convex, and it can be characterized as Ω^∗=_𝑤∈Ω𝑤∈Ω∶(𝑤−𝑤)^𝑇𝐹(𝑤)≥0. (7)

Based on Theorem 1, 𝑤∈Ω can be regarded as an 𝜀-approximation solution of VI(Ω,𝐹) if it satisfies sup_𝑤∈𝒟(𝑤−𝑤)^𝑇𝐹(𝑤)≤𝜀, (8) where 𝒟⊆Ω is some compact set. As Definition  1 in [15], we can take 𝒟=ℬ_Ω(𝑤)∶={𝑤∈Ω∣‖𝑤−𝑤‖≤1}. (9)

In the following, we will give a variational reformulation of (4). It is easy to see that the model (4) can be characterized by a variational inequality problem: find 𝑤^∗=(𝑥^∗,𝑦^∗,𝜆^∗)∈Ω∶=𝒳×𝒴×ℜ^𝑚 such that VI(Ω,𝐹)∶𝑤−𝑤^∗𝑇𝐹𝑤^∗≥0,∀𝑤∈Ω, (10a) where 𝑥𝑦𝜆𝑤=,𝐹(𝑤)=∇𝜃₁(𝑥)−𝐴^𝑇𝜆∇𝜃₂𝑦−𝐵^𝑇𝜆𝐴𝑥+𝐵𝑦−𝑏. (10b) Note that the mapping 𝐹 is monotone since 𝜃₁ and 𝜃₂ are convex. As shown in [11], the ADMM scheme (5a), (5b), and (5c) is identical with the following iterative scheme in a cyclical sense: 𝑥^𝑘∈𝒳,𝑥−𝑥^𝑘𝑇∇𝜃₁𝑥^𝑘−𝐴^𝑇𝜆^𝑘−𝐻𝐴𝑥^𝑘+𝐵𝑦^𝑘𝜆−𝑏≥0,∀𝑥∈𝒳,_𝑘∶=𝜆^𝑘−𝐻𝐴𝑥^𝑘+𝐵𝑦^𝑘,𝑦−𝑏_𝑘∈𝒴,𝑦−𝑦^𝑘𝑇∇𝜃₂𝑦^𝑘−𝐵^𝑇𝜆^𝑘−𝐻𝐴𝑥^𝑘+𝐵𝑦^𝑘−𝑏≥0,∀𝑦∈𝒴, (11a)(11b)(11c)𝑤^𝑘+1=𝑤^𝑘−𝑤^𝑘−𝑤^𝑘. (12) Based on the definition (6) of the matrix 𝐺, we can rewrite (11a), (11b), (11c), and (12) as a special case of the relaxed PPA with 𝛾=1 immediately.

Lemma 2. For given 𝑤^𝑘, let 𝑤^𝑘 be generated by the ADMM scheme (11a), (11b), and (11c). Then, one has 𝑤^𝑘∈Ω,𝑤−𝑤^𝑘𝑇𝐹𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘≥0,∀𝑤∈Ω, (13) where 𝐹 and 𝐺 are defined by (10b) and (6), respectively.

3. The Contraction of the Relaxed Proximal Point Algorithm

In this section, we prove the contraction of the relaxed PPA. First, we give an important lemma.

Lemma 3. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, one has 𝑤−𝑤^𝑘𝑇𝐹𝑤^𝑘≥12𝛾𝑤−𝑤^𝑘+12_𝐺−𝑤−𝑤^𝑘2_𝐺+𝛾1−2𝑤^𝑘−𝑤^𝑘2_𝐺,∀𝑤∈Ω. (14)

Proof. First, using (3a), we have 𝑤−𝑤^𝑘𝑇𝐹𝑤^𝑘≥𝑤−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘,∀𝑤∈Ω. (15) Since 𝑤^𝑘−𝑤^𝑘=(𝑤^𝑘−𝑤^𝑘+1)/𝛾 (see (3b)), we have 𝑤−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘=1𝛾𝑤−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘+1. (16) Thus, it suffices to show that 𝑤−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘+1=12𝑤−𝑤^𝑘+12_𝐺−𝑤−𝑤^𝑘2_𝐺𝛾+𝛾1−2𝑤^𝑘−𝑤^𝑘2_𝐺. (17) By setting 𝑎=𝑤, 𝑏=𝑤^𝑘, 𝑐=𝑤^𝑘, and 𝑑=𝑤^𝑘+1 in the identity (𝑎−𝑏)^𝑇𝐺()=1𝑐−𝑑2‖𝑎−𝑑‖²_𝐺−‖𝑎−𝑐‖²_𝐺+12‖𝑐−𝑏‖²_𝐺−‖𝑑−𝑏‖²_𝐺, (18) we derive that 𝑤−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘+1=12𝑤−𝑤^𝑘+12_𝐺−𝑤−𝑤^𝑘2_𝐺+12𝑤^𝑘−𝑤^𝑘2_𝐺−𝑤^𝑘+1−𝑤^𝑘2_𝐺. (19) On the other hand, using (3b), we have 𝑤^𝑘−𝑤^𝑘2_𝐺−𝑤^𝑘+1−𝑤^𝑘2_𝐺=𝑤^𝑘−𝑤^𝑘2_𝐺−𝑤^𝑘−𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+12_𝐺=𝑤^𝑘−𝑤^𝑘2_𝐺−𝑤^𝑘−𝑤^𝑘−𝛾𝑤^𝑘−𝑤^𝑘2_𝐺=𝛾2−𝛾𝑤^𝑘−𝑤^𝑘2_𝐺. (20) Combining the last two equations, we obtain (17). The assertion (14) follows immediately. The proof is completed.

With the proved lemma, we are now ready to show the contraction of the relaxed PPA (3a) and (3b).

Theorem 4. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, for any 𝑘≥0, one has 𝑤^𝑘+1−𝑤^∗2_𝐺≤𝑤^𝑘−𝑤^∗2_𝐺−𝛾2−𝛾𝑤^𝑘−𝑤^𝑘2_𝐺,∀𝑤^∗∈Ω^∗. (21)

Proof. Setting 𝑤=𝑤^∗ in (14), we get 2𝛾𝑤^∗−𝑤^𝑘𝑇𝐹𝑤^𝑘≥𝑤^∗−𝑤^𝑘+12_𝐺−𝑤^∗−𝑤^𝑘2_𝐺+𝛾2−𝛾𝑤^𝑘−𝑤^𝑘2_𝐺. (22) On the other hand, since 𝐹 is monotone and 𝑤^∗∈Ω^∗, we have 0≥𝑤^∗−𝑤^𝑘𝑇𝐹𝑤^∗≥𝑤^∗−𝑤^𝑘𝑇𝐹𝑤^𝑘. (23) It follows from the previous two inequalities that 𝑤^𝑘+1−𝑤^∗2_𝐺≤𝑤^𝑘−𝑤^∗2_𝐺−𝛾2−𝛾𝑤^𝑘−𝑤^𝑘2_𝐺. (24) The proof is completed.

4. Ergodic Worst-Case 𝑂(1/𝑡) Convergence Rate

In this section, we will establish an ergodic worst-case 𝑂(1/𝑡) convergence rate for the relaxed PPA in the sense that after 𝑡 iterations of such an algorithm, we can find 𝑤∈Ω such that (𝑤−𝑤)^𝑇𝐹(𝑤)≤𝜀,∀𝑤∈ℬ_Ω(𝑤), (25) with 𝜀=𝑂(1/𝑡) and ℬ_Ω(𝑤)∶={𝑤∈Ω∣‖𝑤−𝑤‖_𝐺≤1}.

Theorem 5. Let {𝑤^𝑘} and {𝑤^𝑘} be the sequences generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. For any integer number 𝑡>0, let 𝑤_𝑡1∶=𝑡+1^𝑡_𝑘=0𝑤^𝑘. (26) Then, one has 𝑤_𝑡∈Ω and 𝑤_𝑡−𝑤^𝑇𝐹(𝑤)≤1()𝑤2𝛾𝑡+1⁰−𝑤²_𝐺,∀𝑤∈Ω. (27)

Proof. From (14), we have 𝑤−𝑤^𝑘𝑇𝐹𝑤^𝑘+1𝑤2𝛾^𝑘−𝑤²_𝐺≥1𝑤2𝛾^𝑘+1−𝑤²_𝐺,∀𝑤∈Ω. (28) Since 𝐹 is monotone, from the previous inequality, we have 𝑤−𝑤^𝑘𝑇𝐹(𝑤)+1𝑤2𝛾^𝑘−𝑤²_𝐺≥1𝑤2𝛾^𝑘+1−𝑤²_𝐺,∀𝑤∈Ω. (29) Summing the inequality (29) over 𝑘=0,1,…,𝑡, we obtain (𝑡+1)𝑤−^𝑡_𝑘=0𝑤^𝑘𝑇𝐹(𝑤)+1𝑤2𝛾⁰−𝑤²_𝐺≥1𝑤2𝛾^𝑡+1−𝑤²_𝐺≥0,∀𝑤∈Ω. (30) Since ∑^𝑡_𝑘=01/(𝑡+1)=1, 𝑤_𝑡 is a convex combination of 𝑤⁰,𝑤¹,…,𝑤^𝑡 and thus 𝑤_𝑡∈Ω. Using the notation of 𝑤_𝑡, we derive 𝑤−𝑤_𝑡^𝑇𝐹(𝑤)+1()𝑤2𝛾𝑡+1⁰−𝑤²_𝐺≥0,∀𝑤∈Ω. (31) The assertion (27) follows from the previous inequality immediately.

It follows from Theorem 4 that the sequence {‖𝑤^𝑘‖_𝐺} is bounded. According to (21), the sequence {‖𝑤^𝑘‖_𝐺} is also bounded. Therefore, there exists a constant 𝐷>0 such that 𝑤^𝑘_𝐺≤𝐷,𝑤^𝑘_𝐺≤𝐷,∀𝑘≥0. (32) Recall that 𝑤_𝑡 is the average of {𝑤⁰,𝑤¹,…,𝑤^𝑡}. Thus, we have ‖𝑤_𝑡‖_𝐺≤𝐷. For any 𝑤∈ℬ_Ω(𝑤_𝑡)∶={𝑤∈Ω∣‖𝑤−𝑤_𝑡‖_𝐺≤1}, we get 𝑤_𝑡−𝑤^𝑇𝐹(𝑤)≤1()𝑤2𝛾𝑡+1⁰−𝑤²_𝐺≤1()𝑤2𝛾𝑡+1⁰−𝑤_𝑡𝐺+𝑤_𝑡−𝑤_𝐺²≤1()𝑤2𝛾𝑡+1^0𝐺+𝑤_𝑡𝐺+𝑤_𝑡−𝑤_𝐺²≤(2𝐷+1)²().2𝛾𝑡+1 (33) Thus, for any given 𝜀>0, after at most 𝑡∶=⌈((2𝐷+1)²/2𝛾𝜀)−1⌉ iterations, we have 𝑤_𝑡−𝑤^𝑇𝐹(𝑤)≤𝜀,∀𝑤∈ℬ_Ω𝑤_𝑡, (34) which means that 𝑤_𝑡 is an approximate solution of VI(Ω,𝐹) with an accuracy of 𝑂(1/𝑡). That is, a worst-case 𝑂(1/𝑡) convergence rate of the relaxed PPA in an ergodic sense is established.

Note that this convergence rate is in an ergodic sense and 𝑤_𝑡 is a convex combination of the previous vectors {𝑤⁰,𝑤¹,…,𝑤^𝑡} with equal weights. One may ask if we can establish the same convergence rate in a nonergodic sense directly for the sequence {𝑤^𝑘} generated by the relaxed PPA (3a) and (3b), and this is the main purpose of the next section.

5. Nonergodic Worst-Case 𝑂(1/𝑡) Convergence Rate

This section shows that the relaxed PPA has a worst-case 𝑂(1/𝑡) convergence rate in a nonergodic sense. First, we establish two important inequalities in the following lemmas.

Lemma 6. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, one has 𝑤^𝑘−𝑤^𝑘+1𝑇𝐺𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘+1≥0. (35)

Proof. Setting 𝑤=𝑤^𝑘+1 in (3a), we have 𝑤^𝑘+1−𝑤^𝑘𝑇𝐹𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘≥0. (36) Note that (3a) is also true for 𝑘∶=𝑘+1, and thus we have 𝑤−𝑤^𝑘+1𝑇𝐹𝑤^𝑘+1+𝐺𝑤^𝑘+1−𝑤^𝑘+1≥0,∀𝑤∈Ω. (37) Setting 𝑤=𝑤^𝑘 in the previous inequality, we obtain 𝑤^𝑘−𝑤^𝑘+1𝑇𝐹𝑤^𝑘+1+𝐺𝑤^𝑘+1−𝑤^𝑘+1≥0. (38) Adding (36) and (38) and using the monotonicity of 𝐹, we get (35) immediately.

Lemma 7. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, one has 𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+1≥1𝛾𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+12_𝐺. (39)

Proof. First, adding the term 𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘+1𝑇×𝐺𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘+1 (40) to the both sides of (35), we get 𝑤^𝑘−𝑤^𝑘+1𝑇𝐺𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘+1≥𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘+12_𝐺. (41) Reordering (𝑤^𝑘−𝑤^𝑘+1)−(𝑤^𝑘−𝑤^𝑘+1) in the previous inequality to (𝑤^𝑘−𝑤^𝑘)−(𝑤^𝑘+1−𝑤^𝑘+1), we get 𝑤^𝑘−𝑤^𝑘+1𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+1≥𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+12_𝐺. (42) Substituting the term 𝑤^𝑘−𝑤^𝑘+1=𝛾(𝑤^𝑘−𝑤^𝑘) (see (3b)) into the left-hand side of the last inequality, we obtain (39). The proof is completed.

Next, we prove that {‖𝑤^𝑘−𝑤^𝑘‖_𝐺} is monotonically nonincreasing.

Theorem 8. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, one has 𝑤^𝑘+1−𝑤^𝑘+1_𝐺≤𝑤^𝑘−𝑤^𝑘_𝐺,∀𝑘≥0. (43)

Proof. Setting 𝑎=𝑤^𝑘−𝑤^𝑘 and 𝑏=𝑤^𝑘+1−𝑤^𝑘+1 in the identity ‖𝑎‖²_𝐺−‖𝑏‖²_𝐺=2𝑎^𝑇𝐺(𝑎−𝑏)−‖𝑎−𝑏‖²_𝐺, (44) we obtain 𝑤^𝑘−𝑤^𝑘2_𝐺−𝑤^𝑘+1−𝑤^𝑘+12_𝐺=2𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+12_𝐺. (45) Inserting (39) into the first term of the right-hand side of the last equality and using 𝛾∈(0,2), we obtain 2𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+1−𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+12_𝐺≥2−𝛾𝛾𝑤^𝑘−𝑤^𝑘−𝑤^𝑘+1−𝑤^𝑘+12_𝐺≥0. (46) The assertion (43) follows immediately.

With Theorems 4 and 8, we can prove the worst-case 𝑂(1/𝑡) convergence rate in a nonergodic sense for the relaxed PPA.

Theorem 9. Let the sequences {𝑤^𝑘} and {𝑤^𝑘} be generated by the relaxed PPA (3a) and (3b), and let 𝐺 be a symmetric positive semidefinite matrix. Then, for any integer 𝑡≥0, one has 𝑤^𝑡−𝑤^𝑡2_𝐺≤1𝛾()𝑤2−𝛾𝑡+1⁰−𝑤^∗2_𝐺,∀𝑤^∗∈Ω^∗. (47)

Proof. Summing the inequality (21) over 𝑘=0,1,…,𝑡, we obtain 𝛾2−𝛾^𝑡_𝑘=0𝑤^𝑘−𝑤^𝑘2_𝐺≤𝑤⁰−𝑤^∗2_𝐺−𝑤^𝑡+1−𝑤^∗2_𝐺≤𝑤⁰−𝑤^∗2_𝐺,∀𝑤^∗∈Ω^∗. (48) According to Theorem 8, the sequence {‖𝑤^𝑘−𝑤^𝑘‖_𝐺} is monotonically nonincreasing. Therefore, we have (𝑡+1)𝑤^𝑡−𝑤^𝑡2_𝐺≤^𝑡_𝑘=0𝑤^𝑘−𝑤^𝑘2_𝐺. (49) The assertion (47) follows from (48) and (49) immediately.

Note that Ω^∗ is convex and closed (see Theorem 1). Let 𝑑∶=inf{‖𝑤⁰−𝑤^∗‖_𝐺∣𝑤^∗∈Ω^∗}. Then, for any given 𝜀>0, Theorem 9 shows that the relaxed PPA (3a) and (3b) needs at most ⌈𝑑²/(𝜀𝛾(2−𝛾))−1⌉ iterations to ensure that ‖𝑤^𝑡−𝑤^𝑡‖²_𝐺≤𝜀. Recall that 𝑤^𝑡 is a solution of VI(Ω,𝐹) if ‖𝑤^𝑡−𝑤^𝑡‖²_𝐺=0. In other words, if ‖𝑤^𝑡−𝑤^𝑡‖²_𝐺=0, we have 𝐺(𝑤^𝑡−𝑤^𝑡)=0 since 𝐺 is a positive semidefinite matrix. And thus from (3a), it follows that 𝑤−𝑤^𝑡𝑇𝐹𝑤^𝑡≥0,∀𝑤∈Ω, (50) which means that 𝑤^𝑡 is a solution of VI(Ω,𝐹) according to (1). A worst-case 𝑂(1/𝑡) convergence rate in a nonergodic sense for the relaxed PPA (3a) and (3b) is thus established from Theorem 9.

6. Concluding Remarks

This paper established the worst-case 𝑂(1/𝑡) convergence rate in both ergodic and nonergodic senses for the relaxed PPA. Recall that ADMM is a primal application of the relaxed PPA with 𝛾=1. And thus ADMM also has the same worst-case 𝑂(1/𝑡) convergence rate in both ergodic and nonergodic senses.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant no. 11001053), Program for New Century Excellent Talents in University (Grant no. NCET-12-0111), and Natural Science Foundation of Jiangsu Province, China (Grant no. BK2012662).