An Asymmetric Proximal Decomposition Method for Convex Programming with Linearly Coupling Constraints

Abstract

The problems studied are the separable variational inequalities with linearly coupling constraints. Some existing decomposition methods are very problem specific, and the computation load is quite costly. Combining the ideas of proximal point algorithm (PPA) and augmented Lagrangian method (ALM), we propose an asymmetric proximal decomposition method (AsPDM) to solve a wide variety separable problems. By adding an auxiliary quadratic term to the general Lagrangian function, our method can take advantage of the separable feature. We also present an inexact version of AsPDM to reduce the computation load of each iteration. In the computation process, the inexact version only uses the function values. Moreover, the inexact criterion and the step size can be implemented in parallel. The convergence of the proposed method is proved, and numerical experiments are employed to show the advantage of AsPDM.

1. Introduction

The original model considered here is the convex minimization problem with linearly coupling constraints: minimize^𝑁_𝑗=1𝜃_𝑗𝑥_{𝑗𝑁𝑗=1}𝐴_𝑗𝑥_𝑗−𝑏_𝑗≥0or^𝑁_𝑗=1𝐴_𝑗𝑥_𝑗−𝑏_𝑗,𝑥=0_𝑖∈𝒳_𝑖,𝑖=1,…,𝑁, (1.1) where 𝒳_𝑖⊂ℛ^𝑛𝑖, 𝐴_𝑖 are given 𝑚×𝑛_𝑖 matrixes, 𝑏_𝑖 are given 𝑚-vector, and 𝜃_𝑖∶ℛ^𝑛𝑖→ℛ are the 𝑖th block convex differentiable functions for each 𝑖=1,…,𝑁. This special problem is called convex separable problem. Problems possessing such separable structure arise in discrete-time deterministic optimal control and in the scheduling of hydroelectric power generation [1]. Note that 𝜃_𝑖 are differentiable, setting ∇𝜃_𝑖(𝑥_𝑖)=𝑓_𝑖(𝑥_𝑖); by the well-known minimum principle in nonlinear programming, it is easy to get an equivalent form of problem (1.1): find 𝑥^∗=(𝑥^∗₁,…,𝑥^∗_𝑁)∈Ω such that𝑥^′_𝑖−𝑥^∗_𝑖^𝑇𝑓_𝑖𝑥^∗_𝑖≥0,𝑖=1,…,𝑁,∀𝑥^′∈Ω, (1.2) whereΩ=𝑥₁,…,𝑥_𝑁∣^𝑁_𝑗=1𝐴_𝑗𝑥_𝑗≥𝑏or^𝑁_𝑗=1𝐴_𝑗𝑥_𝑗=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,…,𝑁,𝑏=^𝑁_𝑗=1𝑏_𝑗. (1.3) Problems of this type are called separable variational inequalities (VIs). We will utilize this equivalent formulation and provide method for solution of separable VI.

One of the best-known algorithms for solving convex programming or equivalent VI is the proximal point algorithm (PPA) first proposed by Martinet (see [2]) and had been studied well by Rockafellar [3, 4]. PPA and its dual version, the method of multipliers, draw on a large volume of prior work by various authors [5–9]. However, classical PPA and most of its subsequence papers cannot take advantage of the separability of the original problem, and this makes them inefficient in solving separable structure problems. One major direction of PPA’s study is to develop decomposition methods for separable convex programming and VI. The motivations for decomposition techniques are splitting the problem into isolate smaller or easier subproblems and parallelizing computations on specific parallel computing device. Decomposition-type methods [10–14] for large-scale problems have been widely studied in optimization as well as in variational problems and are explicitly or implicitly derived from PPA. However, most of those methods only can solve separable problems with special equality constraints:Minimize𝜃(𝑥)+𝜓𝑦,𝐴𝑥+𝑦=0,𝑥∈𝒳,𝑦∈𝒴. (1.4) Two very well-known methods for solving equality constrained convex problems and VI are the augmented Lagrangian method [15, 16] (ALM) and the alternating direction method (ADM) [17]. The classic ALM has been deeply studied and has many advantages over the general Lagrange methods; see [18] for more detail. However, it can not preserve separability. ADM is a different method but closely related to ALM, which essentially can preserve separability for problems with two operators (𝑁=2). Recently, separable augmented Lagrangian method (SALM) [19, 20] overcomes the nonseparability of ALM. For example, for solving problem (1.1) with equality constraints, Hamdi and Mahey [19] allocated a resource quantity 𝑦_𝑖 to each block 𝐴_𝑖𝑥_𝑖−𝑏_𝑖 leading (1.1) to an enlarged problem in ∏^𝑁_𝑗=1𝒳_𝑗×ℛ^𝑚𝑁minimize^𝑁_𝑗=1𝜃_𝑗𝑥_{𝑗𝑁𝑗=1}𝑦_𝑗𝐴=0_𝑖𝑥_𝑖−𝑏_𝑖+𝑦_𝑖𝑥=0,𝑖=1,…,𝑁,_𝑖∈𝒳_𝑖,𝑦_𝑖∈ℛ^𝑚,𝑖=1,…,𝑁. (1.5) It is worth mentioning that (1.5) is only equivalent to problem (1.1) with equality constraints. The expression of the augmented lagrangian function of (1.5) is:𝐿𝑥,𝑦,𝑢,𝜏=^𝑁_𝑗=1𝐿_𝑗𝑥_𝑗,𝑦_𝑗,𝑢_𝑗,𝜏, (1.6) with𝐿_𝑗𝑥_𝑗,𝑦_𝑗,𝑢_𝑗,𝜏=𝜃_𝑗𝑥_𝑗−𝑢_𝑗,𝐴_𝑗𝑥_𝑗−𝑏_𝑗+𝑦_𝑗+𝜏2𝐴_𝑗𝑥_𝑗−𝑏_𝑗+𝑦_𝑗². (1.7) SALM finds a saddle point of problem (1.5) by the following stages:(i)𝑥_𝑖^𝑘+1=argmin_{𝑥𝑖∈𝒳𝑖}𝐿_𝑖(𝑥_𝑖,𝑦^𝑘_𝑖,𝑢^𝑘_𝑖,𝜏);(ii)𝑦_𝑖^𝑘+1=argmin_{∑𝑁𝑗=1𝑦𝑗=0}𝐿_𝑖(𝑥_𝑖^𝑘+1,𝑦_𝑖,𝑢^𝑘_𝑖,𝜏);(iii)𝑢_𝑖^𝑘+1=𝑢^𝑘_𝑖+𝜏(𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏_𝑖+𝑦_𝑖^𝑘+1).

Note that the process in SALM for 𝑥^𝑘+1 allows one to solve 𝑁 subproblems in parallel. This has great practical importance from the computation point of view. In fact, SALM belongs to the family of splitting algorithms and ADM for solving special convex problem (1.4) with 𝜓(𝑦)=0 and𝐴𝐴=¹⋱𝐴_𝑁; (1.8) SALM has to introduce an additive variable 𝑦 to exploit the inner separable structure of the problem, which makes the problem larger. Moreover, SALM is suitable to solve equality constraints problems and fraught with difficulties in solving inequality constraints problems.

To our best knowledge, there are few dedicated methods for solving inequality constraints problems (1.1) or VI(1.2)-(1.3), except the decomposition method proposed by Tseng [21] and the PPA-based contraction method by He et al. [22]. The decomposition method in [21] decomposes the computation of 𝑥^𝑘+1 at a fine level without introducing additive variable 𝑦. But, in each iteration of this method, the minimization subproblems for 𝑥^𝑘+1 are dependent on the step size of multiplier, which greatly restricts the computation of subproblems. The PPA-based contraction method in [22] has a nice decomposable structure; however, it has to solve the subproblem exactly. To solve (1.1) or VI(1.2)-(1.3), motivated by PPA-based contraction method and SLAM, we propose an asymmetric proximal decomposition method (AsPDM) which can well conserve the separability feature of the problem. Besides, it does not need to introduce the resource variables 𝑦 like SALM and the subproblems do not depend on the step size of multiplier. In the following, we briefly describe our method for (1.1): we add an auxiliary quadratic term to the general Lagrangian function:𝐿𝑥,𝑥^𝑘_𝑖,𝜆=^𝑁_𝑗=1𝐿_𝑗𝑥_𝑗,𝑥^𝑘_𝑗,𝜆, (1.9) with𝐿_𝑗𝑥_𝑗,𝑥^𝑘_𝑗,𝜆=𝜃_𝑗𝑥_𝑗−𝜆,𝐴_𝑗𝑥_𝑗−𝑏_𝑗+𝛽_𝑗2𝑥_𝑗−𝑥^𝑘_𝑗². (1.10) The general framework of AsPDM is as follows:

Phase I𝑥^𝑘_𝑖=argmin_{𝑥𝑖∈𝒳𝑖}𝐿_𝑖𝑥_𝑖,𝑥^𝑘_𝑖,𝜆^𝑘; (1.11)

Phase II𝜆^𝑘=𝑃_Λ𝜆^𝑘−𝜇^−1𝑁_𝑗=1𝐴_𝑗𝑥^𝑘_𝑗−𝑏_𝑗,Λ=𝑅^𝑚₊or𝑅^𝑚,𝑤^𝑘+1=𝑤^𝑘−𝛼_𝑘𝐺𝑤^𝑘−𝑤^𝑘,𝑤=(𝑥,𝜆). (1.12)

Here, 𝛽_𝑖>0, 𝜇>0, 𝛼_𝑘>0, and 𝐺 are proper chosen which will be detailed in the later sections. Note that the first phase consists of 𝑁 isolate subproblems, and each involves 𝑥_𝑖, 𝑖=1,…,𝑁 only, namely; it can be partitioned into 𝑁 independent lower-dimension subproblems. Hence, this method can take advantage of operators’ separability. Since we mainly focus on solving equivalent separable VI, hence, we present this method under VI framework and analyze its convergence in the following sections.

2. The Asymmetric Proximal Decomposition Method

2.1. Structured VI

The separable VI(1.2)-(1.3) consists of 𝑁 partitioned sub-VIs. Introducing a Lagrange multiplier vector 𝜆∈Λ(Λ=ℛ^𝑚₊orℛ^𝑚) associated with the linearly coupling constraint ∑^𝑁_𝑗=1𝐴_𝑗𝑥_𝑗≥𝑏  (or∑^𝑁_𝑗=1𝐴_𝑗𝑥_𝑗=𝑏), we equivalently formulate the separable VI(1.2)-(1.3) as an enlarged VI: find 𝑤^∗=(𝑥^∗,𝜆^∗)∈𝒲 such that𝑥^′_𝑖−𝑥^∗_𝑖^𝑇𝑓_𝑖𝑥^∗_𝑖−𝐴^𝑇_𝑖𝜆^∗𝜆≥0,𝑖=1,…,𝑁,_′−𝜆^∗𝑇𝑁_𝑗=1𝐴_𝑗𝑥^∗_𝑗−𝑏≥0,∀𝑤′∈𝒲, (2.1) where𝒲=𝒳×Λ,𝒳=^𝑁_𝑗=1𝒳_𝑗. (2.2) VI(2.1)-(2.2) is referred as structured variational inequality (SVI), denoted as SVI(𝒲,𝑄). Here,𝑄(𝑤)=𝑓₁𝑥₁−𝐴^𝑇₁𝜆⋮𝑓_𝑁𝑥_𝑁−𝐴^𝑇_𝑁𝜆_𝑁𝑗=1𝐴_𝑗𝑥_𝑗−𝑏. (2.3)

2.2. Preliminaries

We summarize some basic properties and related definitions which will be used in the following discussions.

Definition 2.1. (i) The mapping 𝑓 is said to be monotone if and only if (𝑤−𝑤)^𝑇𝑓(𝑤)−𝑓(𝑤)≥0,∀𝑤,𝑤. (2.4)
(ii) A function 𝑓 is said to be Lipschitz continuous if there is a constant 𝐿>0 such that 𝑓(𝑤)−𝑓(𝑤)≤𝐿‖𝑤−𝑤‖,∀𝑤,𝑤. (2.5)

The projection onto a closed convex set is a basic concept in this paper. Let 𝒲⊂ℛ^𝑛 be any closed convex set. We use 𝑃_𝒲(𝑤) to denote the projection of 𝑤 onto 𝒲 under the Euclidean norm; that is,𝑃_𝒲(𝑤)=argmin𝑤−𝑤^′∣𝑤^′∈𝒲. (2.6) The following lemmas are useful for the convergence analysis in this paper.

Lemma 2.2. Let 𝒲 be a closed convex set in ℛ^𝑛, then one has (1)𝑤−𝑃_𝒲(𝑤)^𝑇𝑤^′−𝑃_𝒲(𝑤)≤0∀𝑤∈ℛ^𝑛,∀𝑤^′∈𝒲, (2.7)(2)𝑃_𝒲(𝑤)−𝑤^′2≤𝑤−𝑤^′2−𝑤−𝑃_𝒲(𝑤)²,∀𝑤∈ℛ^𝑛,𝑤^′∈𝒲. (2.8)

Proof. See [23].

Lemma 2.3. Let 𝒲 be a closed convex set in ℛ^𝑛, then 𝑤^∗ is a solution of VI(𝒲,𝑄) if and only if 𝑤^∗=𝑃_𝒲𝑤^∗−𝛽𝑄𝑤^∗,∀𝛽>0. (2.9)

Proof. See [10, page 267].

Hence, solving VI(𝒲,𝑄) is equivalent to finding a zero point of the residue function𝑒𝑤,𝛽=𝑤−𝑃_𝒲𝑤−𝛽𝑄(𝑤),𝛽>0. (2.10) Generally, the term ‖𝑒(𝑤)‖ (denotes ‖𝑒(𝑤,1)‖) is referred to as the error bound of VI(𝒲,𝑄), since it measures the distance of 𝑢 from the solution set.

2.3. The Presentation of the Exact AsPDM

In each iteration, by our proper construction, our method solves 𝑁 independent sub-VIs involving each individual variable 𝑥_𝑖 only so that 𝑥_𝑖 can be obtained in parallel. In what follows, to illustrate our method’s practical significance, we interpret our algorithm process as a system which has a central authority and 𝑁 local administrators; each administrator attempts to unilaterally solve a certain problem under the presumption that the instructions given by the authority are parametric inputs and the responses of other administrations’ actions are not available; namely, the 𝑁 local administrators acts synchronously and independently once they receive the information given by the central authority. We briefly describe our method which consists of two main phases.

Phase I: For arbitrary (𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) given by the central authority, each local administrator uses his own way to offer the solution (denoted as 𝑥_𝑖^𝑘) of his individual problem: find 𝑥^𝑘_𝑖∈𝒳_𝑖, such that 𝑥^′_𝑖−𝑥^𝑘_𝑖^𝑇𝑓_𝑖𝑥^𝑘_𝑖+𝛽_𝑖𝑥^𝑘_𝑖−𝑥^𝑘_𝑖−𝐴^𝑇_𝑖𝜆^𝑘≥0,∀𝑥^′_𝑖∈𝒳_𝑖. (2.11)

Phase II: After the 𝑁 local administrators accomplish their tasks, the central authority collects the resulting (𝑥^𝑘₁,…,𝑥^𝑘_𝑁), moreover, corresponding (𝑓₁(𝑥^𝑘₁),…,𝑓_𝑁(𝑥^𝑘_𝑁)) which can be viewed as the feedback information from the 𝑁 local administrators and sets: 𝜆^𝑘=𝑃_Λ𝜆^𝑘−𝜇^−1𝑁_𝑗=1𝐴_𝑗𝑥^𝑘_𝑗−𝑏. (2.12) Here, 𝜇 is suitably chosen by the central authority. So the central authority aims to employ this feedback information effectively to provide (𝑥₁^𝑘+1,…,𝑥_𝑁^𝑘+1,𝜆^𝑘+1) which will be beneficial for the next iteration loop. In this paper, our proposed methods will update the new iterate by the following two forms: 𝑤₁^𝑘+1=𝑤^𝑘−𝛼_𝑘𝐺𝑤^𝑘−𝑤^𝑘, (2.13) or 𝑤₂^𝑘+1=𝑃_𝒲𝑤^𝑘−𝛼_𝑘𝑄𝑤^𝑘, (2.14) where 𝛼_𝑘>0 is a specific step size and 𝛽𝐺=¹𝐴^𝑇₁𝛽⋱⋮_𝑁𝐴^𝑇_𝑁𝜇𝐼. (2.15)

We make the standard assumptions to guarantee that the problem under consideration is solvable and the proposed methods are well defined.

Assumption A. 𝑓_𝑖(𝑥_𝑖) is monotone and Lipschitz continuous, 𝑖=1,…,𝑁.

By this assumption, it is easy to get that 𝑄(𝑤) is monotone.

2.4. Presentation of the Inexact AsPDM

In this section, the inexact version of the AsPDM method is present, and some remarks are briefly made.

For later analysis convenience, we denote𝛽𝐷=¹2⋱𝛽_𝑁2𝜂𝐼. (2.16) At each iteration, we solve 𝑁 sub-VIs (see (2.11)) independently. No doubt, the computation load for an exact solution of (2.11) is usually expensive. Hence, it is desirable for us to consider solving (2.11) inexactly under a relative relaxed inexact criterion. We now describe and analyze our inexact method. Each iteration consists of two main phases, one of which provides an inexact solution of (2.11) and the other of which employs the inexact solution to offer a new iterate for the next iteration.

The first phase of our method works as follows. At the beginning of 𝑘th iteration, an iterate 𝑤^𝑘=(𝑥^𝑘,𝜆^𝑘) is given. If 𝑥^𝑘_𝑖=𝑃_𝒳𝑖[𝑥^𝑘_𝑖−(𝑓_𝑖(𝑥^𝑘_𝑖)−𝐴^𝑇_𝑖𝜆^𝑘)], then 𝑥^𝑘_𝑖 is the exact solution of 𝑖th sub-VI; there is nothing we need to do with the 𝑖th sub-VI. Otherwise, we should find 𝑥_𝑖^𝑘∈𝒳_𝑖 such that𝑥^′_𝑖−𝑥^𝑘_𝑖^𝑇𝑓_𝑖𝑥^𝑘_𝑖+𝛽_𝑖𝑥^𝑘_𝑖−𝑥^𝑘_𝑖−𝐴^𝑇_𝑖𝜆^𝑘+𝜉^𝑘_𝑥𝑖≥0,∀𝑥^′_𝑖∈𝒳_𝑖, (2.17) with𝜉^𝑘_𝑥𝑖=𝑓_𝑖𝑥^𝑘_𝑖−𝑓_𝑖𝑥^𝑘_𝑖. (2.18) Here, the obtained 𝛽_𝑖 should satisfy following two inexact criteria:𝑥^𝑘_𝑖−𝑥^𝑘_𝑖^𝑇𝜉^𝑘_𝑥𝑖≤𝜈𝛽_𝑖2𝑥^𝑘_𝑖−𝑥^𝑘_𝑖²,𝜈∈(0,1), (2.18∗a)𝜉^𝑘_𝑥𝑖≤𝛽_𝑖√2𝑥^𝑘_𝑖−𝑥^𝑘_𝑖. (2.18∗b)

Once one of the above criteria fails to be satisfied, we will increase 𝛽_𝑖 by 𝛽_𝑖=𝛽_𝑖∗1.8 and turn back to solve the 𝑖th sub-VI of (2.17) with this updated 𝛽_𝑖. It should be noted that both inexact criteria are quite easy to check since they do not contain any unknown variables. In addition, another favorable characterization of these criteria is that they are independent; namely, they only involve 𝑥^𝑘_𝑖,𝑥^𝑘_𝑖, irrelevant to 𝑥^𝑘_𝑗,𝑥^𝑘_𝑗(𝑗≠𝑖).

In what follows, let us describe the second phase. We require𝜆^𝑘=𝑃_Λ𝜆^𝑘−𝜇^−1𝑁_𝑗=1𝐴_𝑗𝑥^𝑘_𝑗−𝑏, (2.19) where 𝜇∈(0,∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂] (here, 𝜂>0) is suitably chosen to satisfy𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘≥𝑤^𝑘−𝑤^𝑘2_𝐷. (2.20) Now we use this 𝑤^𝑘=(𝑥^𝑘,𝜆^𝑘) (or 𝑄(𝑤^𝑘)) to construct the new iteration. Here, we provide two simple forms for the new iteration:𝑤₁^𝑘+1=𝑤^𝑘−𝛼_𝑘𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘,𝜉^𝑘=𝜉^𝑘_𝑥,0,𝜉^𝑘_𝑥=𝜉^𝑘_𝑥1,…,𝜉^𝑘_𝑥𝑁, (2.21∗a) or𝑤₂^𝑘+1=𝑃_𝒲𝑤^𝑘−𝛼_𝑘𝑄𝑤^𝑘, (2.21∗b) where𝛼_𝑘=𝛾𝛼^∗_𝑘,𝛼^∗_𝑘=𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉²,𝛾∈(0,2). (2.21) In fact, each iteration of the proposed method consists of two main phases. Using the point of view that the problem is a system with a central authority and 𝑁 administrators, the first phase is accomplished by 𝑁 administrators based on the instruction given by the authority. That is, 𝑖th sub-VI only involves 𝑖th administrator’s activities. On the other hand, the second phase is implemented by the central authority to give new instruction for the next iteration.

Remark 2.4. In the inexact AsPDM, the main task of Phase I is to find a solution for (2.17). From (2.17), it is easy to get that 𝑥^𝑘_𝑖=𝑃_𝒳𝑖𝑥^𝑘_𝑖−𝑓_𝑖𝑥^𝑘_𝑖+𝛽_𝑖𝑥^𝑘_𝑖−𝑥^𝑘_𝑖−𝐴^𝑇_𝑖𝜆^𝑘. (2.22) It seems that equality (2.22) is an implicit form since both sides of (2.22) contain 𝑥^𝑘_𝑖. In fact, we can transform equality (2.22) to an explicit form. Using the property of the projection, we have 𝑥^𝑘_𝑖=𝑃_𝒳𝑖𝑥^𝑘_𝑖−𝛽_𝑖𝑥^𝑘_𝑖−𝑥^𝑘_𝑖+𝛽_𝑖⁻¹𝑓_𝑖𝑥^𝑘_𝑖−𝐴^𝑇_𝑖𝜆^𝑘=𝑃_𝒳𝑖𝑥^𝑘_𝑖−1𝛽_𝑖𝑓_𝑖𝑥^𝑘_𝑖−𝐴^𝑇_𝑖𝜆^𝑘. (2.23) Consequently, using the above formula, we can compute 𝑥^𝑘_𝑖 quite easily.

Remark 2.5. Combining (2.22) and (2.19), we then find that 𝑤^𝑘=𝑃_𝒲𝑤^𝑘−𝑄𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘+𝜉^𝑘. (2.24) If 𝜉^𝑘=0, it yields an exact version. In this special case, it is clear that 𝑤^𝑘=𝑃_𝒲𝑤^𝑘−𝑄𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘. (2.25) We find that this formula is quite similar to the iterates produced by the classic PPA [3], which employs 𝑤^𝑘+1=𝑃_𝒲𝑤^𝑘+1−𝑄𝑤^𝑘+1+𝑆𝑤^𝑘+1−𝑤^𝑘 (2.26) as the new iterate; here, 𝑆 is a positive symmetry definite matrix. For deeper insight, our method does not appear fit into any of the known PPA frameworks. It is virtually not equivalent to PPA even if 𝐺 is positive definite. The reason why our method can not be viewed as PPA lies in the fact that 𝐺 is asymmetry, moreover, may be not positive definite. This lack of symmetry makes it fail to introduce an inner product as 𝑆. Consequently, if one sets 𝑤^𝑘 as the new iterate, one may fail to obtain the convergence. Due to the asymmetric feature of 𝐺, we call our method asymmetric proximal decomposition method.

Remark 2.6. Recalling that 𝜆^𝑘 is obtained by (2.19), it is easy to get that 𝜆^′−𝜆^𝑘𝑇𝜆^𝑘−𝜆^𝑘+𝜇^−1𝑁_𝑗=1𝐴_𝑗𝑥_𝑗−𝑏≥0,∀𝜆^′∈ℛ^𝑚₊. (2.27) Combining (2.17) and (2.27), we have 𝑤^′−𝑤^𝑘𝑇𝑄𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘+𝜉^𝑘≥0,∀𝑤^′∈𝒲. (2.28)
Since 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘)∈𝒲 is generated by (2.17)–(2.20) from a given (𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘), we have that ‖𝑤^𝑘−𝑤^𝑘‖=0 implies ‖𝜉^𝑘‖=0 and ‖𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘‖=0. According to (2.28), we have 𝑤^′−𝑤^𝑘𝑇𝑄𝑤^𝑘≥0∀𝑤^′∈𝒲. (2.29) In other words, 𝑤^𝑘 is a solution of Problem (2.1)-(2.2) if 𝑥^𝑘_𝑖=𝑥^𝑘_𝑖 (𝑖=1,…,𝑁) and 𝜆^𝑘=𝜆^𝑘. Hence, we use ‖𝑤^𝑘−𝑤^𝑘‖≤𝜀 as stopping criterion in the proposed method.

Remark 2.7. The update form (2.21*a) is based on the fact that 𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘 is a descent direction of the unknown distance function (1/2)‖𝑤−𝑤^∗‖² at point 𝑤^𝑘. This property will be proved in Section 3.1. 𝛼^∗_𝑘 in (2.21) is the “optimal” step length, which will be detailed in Section 3.2. We can also use (2.21*b) to update the new iterate. For fast convergence, the practical step length should be multiplied by a relaxed factor 𝛾∈[1,2).

Remark 2.8. Note that ‖𝑤^𝑘−𝑤^𝑘‖=0 if and only if ‖𝑤^𝑘−𝑤^𝑘‖_𝐷=0. In the case ‖𝑤^𝑘−𝑤^𝑘‖_𝐷≠0, by choosing a suitable 𝜇∈(0,∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂], (2.20) will be satisfied. We state this fact in the following lemma.

Lemma 2.9. Let 𝐺 and 𝐷 be defined in (2.15) and (2.16), respectively. If 𝜇=∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂, for all 𝑤=(𝑥₁,…,𝑥_𝑁,𝜆)∈ℛ^{∑𝑁𝑗=1𝑛𝑗+𝑚}, one has 𝑤^𝑇𝐺𝑤≥‖𝑤‖²_𝐷. (2.30)

Proof. According to Definition (2.16), we have 𝑤^𝑇𝐺𝑤=^𝑁_𝑗=1𝛽_𝑗𝑥_𝑗²+^𝑁_𝑗=1𝐴_𝑗𝐴^𝑇_𝑗2𝛽_𝑗+𝜂‖𝜆‖²+^𝑁_𝑗=1𝑥^𝑇_𝑗𝐴^𝑇_𝑗𝜆=^𝑁_𝑗=1𝛽_𝑗2𝑥_𝑗²+𝜂‖𝜆‖²+12^𝑁_𝑗=1𝛽_𝑗𝑥_𝑗+𝐴^𝑇_𝑗𝜆𝛽_𝑗²+^𝑁_𝑗=1𝐴_𝑗𝐴^𝑇_𝑗‖𝜆‖²−𝐴^𝑇_𝑗𝜆²2𝛽_𝑗≥‖𝑤‖²_𝐷, (2.31) and the assertion is obtained.

Set 𝑤=𝑤^𝑘−𝑤^𝑘 in the above lemma, we get𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘≥𝑤^𝑘−𝑤^𝑘2_𝐷. (2.32) If one chooses 𝜇=∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂, then the Condition (2.20) is always satisfied; hence, ∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂 can be regarded as a safe upper bound for this condition. Note that we use an inequality in the proof of Lemma 2.9; it seems that there exists some relaxations. As a result, rather than fix 𝜇=∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂, let 𝜇 be a smaller value, and check if Condition (2.20) is satisfied. If not, increase 𝜇 by 𝜇=min{∑^𝑁_𝑗=1(‖𝐴_𝑗𝐴^𝑇_𝑗‖/2𝛽_𝑗)+𝜂,4𝜇} and try again. This process enables us to reach a suitable 𝜇∈(0,(∑^𝑁_𝑗=1‖𝐴_𝑗𝐴^𝑇_𝑗‖+1)/2] to meet (2.20).

Note that, in our proposed method, problems VI (2.17) produce 𝑥^𝑘_𝑖 in a parallel wise. In addition, instead of taking the solution of the subproblems, the new iterate in the proposed methods is updated by a simple manipulation, for example, (2.18*a)-(2.18*b).

3. Convergence of AsPDM

In the proposed methods, the first phase (accomplished by the local administrators) offers a descent direction of the unknown distance function, and the second phase (accomplished by the central authority) determines the “optimal” step length along this direction. This section gives more theory analysis.

3.1. The Descent Direction in the Proposed AsPDM

For any 𝑤^∗∈𝒲^∗, (𝑤^𝑘−𝑤^∗) is the gradient of the unknown distance function (1/2)‖𝑤−𝑤^∗‖² at point 𝑤^𝑘∉𝒲^∗. A direction 𝑑 is called a descent direction of (1/2)‖𝑤−𝑤^∗‖² at point 𝑤^𝑘 if and only if the inner product ⟨𝑤^𝑘−𝑤^∗,𝑑⟩<0. Let 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) be generated by (2.17)–(2.20) from a given 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘). A goal of this subsection is to elucidate that, for any 𝑤^∗∈𝒲^∗,𝑤^𝑘−𝑤^∗𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥(1−𝜈)𝑤^𝑘−𝑤^𝑘2_𝐷. (3.1) It guarantees that 𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘 is a descent direction of (1/2)‖𝑤−𝑤^∗‖² at point 𝑤^𝑘∉𝒲^∗. The above inequality plays an important role in the convergence analysis.

Lemma 3.1. Assume that 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) is generated by (2.17)–(2.20) from a given 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘), then for any 𝑤^∗=(𝑥^∗₁,…,𝑥^∗_𝑁,𝜆^∗)∈𝒲^∗ one has 𝑤^𝑘−𝑤^∗𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥(1−𝜈)𝑤^𝑘−𝑤^𝑘2_𝐷. (3.2)

Proof. Since 𝑤^∗∈𝒲, substituting 𝑤′=𝑤^∗ in (2.28), we obtain 𝑤^∗−𝑤^𝑘𝑇𝑄𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘+𝜉^𝑘≥0. (3.3) Using the monotonicity of 𝑄(𝑤) and applying with 𝑤′=𝑤^𝑘 in (2.1), it is easy to get 𝑤^𝑘−𝑤^∗𝑇𝑄𝑤^𝑘≥𝑤^𝑘−𝑤^∗𝑇𝑄𝑤^∗≥0. (3.4) Combining (3.3) and (3.4), we then find 𝑤^𝑘−𝑤^∗𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘. (3.5) Note that Criterion (2.18*a) holds; we have 𝑤^𝑘−𝑤^𝑘𝑇𝜉^𝑘≤𝜈^𝑁_𝑗=1𝛽_𝑗2𝑥^𝑘_𝑗−𝑥^𝑘_𝑗²≤𝜈𝑤^𝑘−𝑤^𝑘2_𝐷≤𝜈𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘. (3.6) The last inequality follows directly from the result of Lemma 2.9. Consequently, 𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥(1−𝜈)𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘. (3.7) Using the preceding inequality and (2.32) in (3.5) yields 𝑤^𝑘−𝑤^∗𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥(1−𝜈)𝑤^𝑘−𝑤^𝑘2_𝐷, (3.8) completing the proof.

Now, we state the main properties of 𝑄(𝑤^𝑘) in the lemma below.

Lemma 3.2. Let 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) be generated by (2.17)–(2.20) from given 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘). Then for any 𝑤^′∈𝒲 one has 𝑤^′−𝑤^𝑘𝑇𝑄𝑤^𝑘≥𝑤^′−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘. (3.9)

Proof. Recalling that 𝑤^𝑘=𝑃_𝒲[𝑤^𝑘−(𝑄(𝑤^𝑘)+𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘)]∈𝒲, substituting 𝑤=𝑤^𝑘−(𝑄(𝑤^𝑘)+𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘) in inequality (2.1), we have, immediately, 𝑤^′−𝑤^𝑘𝑇𝑤^𝑘−𝑤^𝑘−𝑄𝑤^𝑘+𝐺𝑤^𝑘−𝑤^𝑘+𝜉^𝑘≥0. (3.10) By some manipulations, our assertion holds immediately.

3.2. The Step Size and the New Iterate

Since 𝐺(𝑤^𝑘−𝑤^𝑘)+𝜉^𝑘 is a descent direction of (1/2)‖𝑤−𝑤^∗‖² at point 𝑤^𝑘, the new iterate will be determined along this direction by choosing a suitable step length. In order to explain why we have the “optimal” step 𝛼^∗_𝑘 as defined in (2.21), we let𝑤₁^𝑘+1(𝛼)=𝑤^𝑘−𝛼𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘,𝑤₂^𝑘+1(𝛼)=𝑃_𝒲𝑤^𝑘−𝛼𝑄𝑤^𝑘 (3.11) be the step-size-dependent new iterate, and letΘ_𝑘,𝑖(𝛼)=𝑤^𝑘−𝑤^∗2−𝑤_𝑖^𝑘+1(𝛼)−𝑤^∗2,𝑖=1,2, (3.12) be the profit function of the 𝑘th iteration. Because Θ_𝑘,𝑖(𝛼) includes the unknown vector 𝑤^∗, it can not be maximized directly. The following lemma offers us a lower bound of Θ_𝑘,𝑖(𝛼) which is a quadratic function of 𝛼.

Lemma 3.3. Let 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) be generated by (2.17)–(2.20) from a given 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘). Then one has Θ_𝑘,𝑖(𝛼)≥𝑞_𝑘(𝛼),∀𝛼>0,𝑖=1,2, (3.13) where 𝑞_𝑘(𝛼)=2𝛼𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘−𝛼²𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2. (3.14)

Proof. It follows from Definition (3.12) and inequality (3.5) that Θ_𝑘,1(𝛼)=𝑤^𝑘−𝑤^∗2−𝑤^𝑘−𝑤^∗−𝛼𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘2=2𝛼𝑤^𝑘−𝑤^∗𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘−𝛼²𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2_(3.5)≥𝑞_𝑘(𝛼). (3.15) Let us deal with Θ_𝑘,2(𝛼) which seems more complicated: Θ_𝑘,2(𝛼)≥^(2.8)𝑤^𝑘−𝑤^∗2−𝑤^𝑘−𝑤^∗−𝛼_𝑘𝑄(𝑤^𝑘)²+𝑤^𝑘−𝑤^𝑘+1−𝛼_𝑘𝑄(𝑤^𝑘)²=2𝛼𝑤^𝑘+1−𝑤^∗𝑇𝑄𝑤^𝑘+𝑤^𝑘−𝑤^𝑘+12≥2𝛼𝑤^𝑘+1−𝑤^𝑘𝑇𝑄𝑤^𝑘+𝑤^𝑘−𝑤^𝑘+12_(3.9)≥𝑤^𝑘−𝑤^𝑘+12+2𝛼𝑤^𝑘+1−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘=𝑤^𝑘−𝑤^𝑘+1−𝛼𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘2+2𝛼𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘−𝛼²𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2≥2𝛼𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘−𝛼²𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2. (3.16) Since 𝑞_𝑘(𝛼) is a quadratic function of 𝛼, it reaches its maximum at 𝛼^∗_𝑘=𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2; (3.17) this is just the same defined in (2.21). In practical computation, taking a relaxed factor 𝛾 is wise for fast convergence. Note that for any 𝛼_𝑘=𝛾𝛼^∗_𝑘, it follows from (3.13), (3.14), and (2.21) that Θ_𝑘,𝑖𝛾𝛼^∗_𝑘≥𝑞_𝑘𝛾𝛼^∗_𝑘=2𝛾𝛼^∗_𝑘𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘−𝛾²𝛼^∗_𝑘𝛼^∗_𝑘𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘2=𝛾2−𝛾𝛼^∗_𝑘𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘. (3.18) In order to guarantee that the right-hand side of (3.18) is positive, we take 𝛾∈(0,2).

In fact, 𝛼^∗_𝑘 is bounded below by a positive amount which is the subject of the following lemma.

Lemma 3.4. Assume that 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘) is generated by (2.17)–(2.20) from a given 𝑤^𝑘=(𝑥^𝑘₁,…,𝑥^𝑘_𝑁,𝜆^𝑘), then one has 𝛼^∗_𝑘≥1−𝜈𝐺𝐷^−1/2+1². (3.19)

Proof. Using the fact that square matrix 𝐷 is positive symmetric definite, we have 𝐺𝑤^𝑘−𝑤^𝑘=𝐺𝐷^−1/2𝐷^1/2𝑤^𝑘−𝑤^𝑘≤𝐺𝐷^−1/2𝐷^1/2𝑤^𝑘−𝑤^𝑘. (3.20) Moreover, Note that Criterion (2.18*b) implies 𝜉^𝑘2≤^𝑁_𝑗=1𝛽_𝑗2𝑥^𝑘_𝑗−𝑥^𝑘_𝑗²+𝜂𝜆^𝑘−𝜆^𝑘2=𝑤^𝑘−𝑤^𝑘2_𝐷. (3.21) Hence, applying ‖𝑤^𝑘−𝑤^𝑘‖²_𝐷=‖𝐷^1/2(𝑤^𝑘−𝑤^𝑘)‖² in the above inequality, we get 𝜉^𝑘≤𝐷^1/2𝑤^𝑘−𝑤^𝑘. (3.22) Combining (3.20) and (3.22), we have 𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≤𝐺𝐷^−1/2+1𝐷^1/2𝑤^𝑘−𝑤^𝑘. (3.23) Consequently, applying the above inequality, (3.7), and (2.32) to 𝛼^∗_𝑘 yields 𝛼^∗_𝑘=𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘𝐺(𝑤^𝑘−𝑤^𝑘)−𝜉^𝑘2≥(1−𝜈)𝑤^𝑘−𝑤^𝑘2_𝐷𝐺𝐷^−1/2+1²𝐷^1/2(𝑤^𝑘−𝑤^𝑘)²≥1−𝜈𝐺𝐷^−1/2+1², (3.24) and thus the assertion is proved.

Now, we are in the stage to prove the main convergence theorem of this paper.

Theorem 3.5. For any 𝑤^∗=(𝑥^∗,𝜆^∗)∈𝒲^∗, the sequence {𝑤^𝑘=(𝑥^𝑘,𝜆^𝑘)} generated by the proposed method satisfies 𝑤^𝑘+1−𝑤^∗2≤𝑤^𝑘−𝑤^∗2−𝛾2−𝛾(1−𝜈)²𝐺𝐷^−1/2+1²𝑤^𝑘−𝑤^𝑘2_𝐷. (3.25) Thus we have lim_𝑘→∞𝑤^𝑘−𝑤^𝑘2=0, (3.26) and the iteration of the proposed method will terminate in finite loops.

Proof. First, it follows from (3.12) and (3.18) that 𝑤^𝑘+1−𝑤^∗2≤𝑤^𝑘−𝑤^∗2−𝛾2−𝛾𝛼^∗_𝑘𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘. (3.27) Using (2.32), (3.7), and (3.19), we have 𝛼^∗_𝑘𝑤^𝑘−𝑤^𝑘𝑇𝐺𝑤^𝑘−𝑤^𝑘−𝜉^𝑘≥(1−𝜈)²𝐺𝐷^−1/2+1²𝑤^𝑘−𝑤^𝑘2_𝐷. (3.28) Substituting (3.28) in (3.27), Assertion (3.25) is proved. Therefore, we have ^∞_𝑘=0𝛾2−𝛾(1−𝜈)²𝐺𝐷^−1/2+1²𝑤^𝑘−𝑤^𝑘2_𝐷≤𝑤⁰−𝑤^∗2, (3.29) and Assertion (3.26) follows immediately.
Since we use max𝑥^𝑘₁−𝑥^𝑘_1∞,…,𝑥^𝑘_𝑁−𝑥^𝑘_𝑁∞,𝜆^𝑘−𝜆^𝑘_∞<𝜖, (3.30) as the stopping criterium, it follows from (3.26) that the iteration will terminate in finite loops for any given 𝜖>0.

4. Numerical Examples

This section describes experiments testifying to the good performance of proposed method. The algorithms were written in Matlab (version 7.0) and complied on a notebook with CPU of Intel Core 2 Duo (2.01 GHz and RAM of 0.98 GB).

To evaluate the behavior of the proposed method, we construct examples about convex separable quadratic programming (CSQP) with linearly coupling constraints. The convex separable quadratic programming was generated as follows:1min2^𝑁_𝑖=1𝑥^𝑇_𝑖𝐻_𝑖𝑥_𝑖−𝑐^𝑇_𝑖𝑥_𝑖∣𝑥∈Ω,Ω=𝑥≥0∣^𝑁_𝑖=1𝐴_𝑖𝑥_𝑖≤𝑏, (4.1) where 𝐻_𝑖∈𝑅^{𝑛𝑖×𝑛𝑖} is a symmetric positive definite matrix, 𝐴_𝑖∈𝑅^{𝑚×𝑛𝑖}, 𝑏∈𝑅^𝑚, and 𝑐_𝑖∈𝑅^𝑛𝑖. We construct matrices 𝐴_𝑖 and 𝐻_𝑖 in the test examples as follows. The elements of 𝐴_𝑖 are randomly given in (−5,5), and the matrices 𝐻_𝑖 are defined by setting𝐻_𝑖∶=𝑉_𝑖Σ_𝑖𝑉^𝑇_𝑖, (4.2) where𝑉_𝑖=𝐼_𝑛𝑖−2𝑣_𝑖𝑣^𝑇_𝑖𝑣^𝑇_𝑖𝑣_𝑖,Σ_𝑖=diag𝜎_𝑘,𝑖,𝜎_𝑘,𝑖=cos𝑘𝜋𝑛_𝑖+1+1. (4.3) In this way, 𝐻_𝑖 is positive definite and has prescribed eigenvalues between (0,2). If 𝑥^∗ is the solution of Problem (4.1), according to the KKT principle, there is a 0≤𝑦^∗∈𝑅^𝑚 such that𝐻_𝑖𝑥^∗_𝑖+𝐴^𝑇_𝑖𝑦^∗−𝑐_𝑖≥0,𝑥^∗𝑇𝐻_𝑖𝑥^∗_𝑖+𝐴^𝑇_𝑖𝑦^∗−𝑐_𝑖=0,𝑥^∗_𝑖≥0,_𝑁𝑖=1𝐴_𝑖𝑥^∗_𝑖≤𝑏,𝑦^∗𝑇𝑁_𝑖=1𝐴_𝑖𝑥^∗_𝑖−𝑏=0,𝑦^∗≥0. (4.4) Let 𝜉_𝑖∈𝑅^𝑛𝑖 and 𝑧∈𝑅^𝑚 be random vectors whose elements are between (−1,1). We set𝑥^∗_𝑖=max𝜉_𝑖,0∗𝜏₁,𝜉^∗_𝑖=max−𝜉_𝑖,0∗𝜏₂,𝑦^∗=max(𝑧,0)∗𝜏₃,𝑧^∗=max(−𝑧,0)∗𝜏₄, (4.5) where 𝜏_𝑖,𝑖=1,2,3,4, are positive parameters. By setting𝑐_𝑖=𝐻_𝑖𝑥^∗_𝑖+𝐴^𝑇_𝑖𝑦^∗−𝜉^∗_𝑖,𝑏=^𝑁_𝑖=1𝐴_𝑖𝑥^∗_𝑖+𝑧^∗, (4.6) we constructed a test problem of (4.1) which has the known solution point 𝑥^∗ and the optimal Lagrangian multipliers 𝑦^∗. We tested such problems with 𝜏₁=0.5,𝜏₂=10,𝜏₃=0.5, 𝜏₄=10. Here, two example sets were considered. The problems in the first set have 3 separable operators (𝑁=3), and the second have 2 (𝑁=2).

In the first experiment, we employ AsPDM with update 𝑤₂^𝑘+1 to solve CSQP with 3 separable operators. (The reason why we choose 𝑤₂^𝑘+1 here is that it usually performs better than 𝑤₁^𝑘+1.) The stopping criterion was chosen as ‖𝑒(𝑤)‖_∞<10⁻³; the parameters were set as 𝜈=0.2, 𝜂=0.5, and 𝛾=1.8. Table 1 reports the number iterations (denoted as Its.), the total number of function evaluations (denoted as Num_𝑓) for different problem-sizes. Here, Num_𝑓=∑³_𝑖=1Num_𝑓𝑖. Observed form Table 1, the solutions are obtained in a moderate number of iterations; thus the proposed method is effectively applicable. In addition, the evaluations of 𝑓_𝑖 per iteration are approximately equal to 2. AsPDM is well suited to solve separable problems.

Table 1 

AsPDM for CSQP with 3 separable operators.

Next, we compared the computational efficiency of AsPDM against the method in [7] (denoted as PCM), regarded as a highly efficient PPA-based method that can be well suited to solve VI. Iterations were terminated when the criterion ‖𝑒(𝑤)‖_∞<10⁻⁵ was met. Table 2 reports the iterations, the total number of function evaluation for both methods. We observe that both methods are acceptable to for us to find a solution. Concerning computational efficiency, we can observe that AsPDM is comparable and clearly faster than PCM; moreover, function evaluations are also less, except in the case of 𝑚=500,𝑛₁=500,𝑛₂=500. In some cases, AsPDM can reduce about 20% computation cost than PCM. For 𝑚=100,𝑛₁=100,𝑛₂=100, we plot the error versus iteration number for both AsPDM and PCM in Figure 1. We have found that both methods converge quickly for the first hundred iterations but slow down as the exact solution is reached. The speed of AsPDM is better than PCM.

Table 2 

Its. and function eval. for different problem sizes.

Figure 1 

Error versus iteration number for the method and for with 𝑚=100, 𝑛1=100, 𝑛2=100.

In addition to being fast, AsPDM can solve the problem separately; that is the most significant advantage over other methods. Hence, AsPDM is more suitable to solve the real-life separable problems.

5. Conclusions

We have proposed AsPDM for solving separable problems. It decomposes the original problem to independent low-dimension subproblems and solves those subproblems in parallel. Only the function values is required in the process, and the total computational cost is very small. AsPDM is easy to implement and does not appear to require application-specific tuning. The numerical results also evidenced the efficiency of our method. Thus, the new method is applicable and recommended in practice.

Acknowledgment

The author was supported by the NSFC Grant 70901018.