Convergence Analysis of Alternating Direction Method of Multipliers for a Class of Separable Convex Programming

Abstract

The purpose of this paper is extending the convergence analysis of Han and Yuan (2012) for alternating direction method of multipliers (ADMM) from the strongly convex to a more general case. Under the assumption that the individual functions are composites of strongly convex functions and linear functions, we prove that the classical ADMM for separable convex programming with two blocks can be extended to the case with more than three blocks. The problems, although still very special, arise naturally from some important applications, for example, route-based traffic assignment problems.

1. Introduction

In this paper, we consider the convex programming with separable functions: min^𝑚_𝑖=1𝑓_𝑖𝑥_𝑖∣^𝑚_𝑖=1𝐴_𝑖𝑥_𝑖=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,2,…,𝑚, (1) where 𝑓_𝑖∶ℛ^𝑛𝑖→ℛ∪{+∞} (𝑖=1,2,…,𝑚) are closed proper convex functions (not necessarily smooth); 𝐴_𝑖∈ℛ^{𝑙×𝑛𝑖} (𝑖=1,2,…,𝑚); 𝒳_𝑖⊆ℛ^𝑛𝑖 (𝑖=1,2,…,𝑚) are closed convex sets; 𝑏∈ℛ^𝑙 and ∑^𝑚_𝑖=1𝑛_𝑖=𝑛. Throughout the paper, we assume that the solution set of (1) is nonempty.

For the special case of (1) with 𝑚=2, min𝑓₁𝑥₁+𝑓₂𝑥₂∣𝐴₁𝑥₁+𝐴₂𝑥₂=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,2, (2) the problem has been studied extensively. Among lots of numerical methods, one of the most popular methods is the alternating direction method of multipliers (ADMM) which was presented originally in [1, 2]. The iterative scheme of ADMM for (2) is as follows: 𝑥₁^𝑘+1=argmin𝑓₁𝑥₁−𝜆^𝑘𝑇𝐴₁𝑥₁+𝛽2𝐴₁𝑥₁+𝐴₂𝑥^𝑘₂−𝑏²∣𝑥₁∈𝒳_𝑖;𝑥₂^𝑘+1=argmin𝑓₂𝑥₂−𝜆^𝑘𝑇𝐴₂𝑥₂+𝛽2𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂−𝑏²∣𝑥₂∈𝒳₂;𝜆^𝑘+1=𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1−𝑏, (3) where 𝜆^𝑘 is Lagrange multiplier associated with the linear constraints and 𝛽>0 is the penalty parameter. The convergence of ADMM for (2) was also established under the condition that the involved functions are convex and the constrained sets are convex too.

While there are diversified applications whose objective function is separable into 𝑚≥3 individual convex functions without coupled variables, such as traffic problems, the problem of recovering the low-rank, sparse components of matrices from incomplete and noisy observation in [3], the constrained total-variation image restoration and reconstruction problem in [4, 5], and the minimal surface PDE problem in [6], it is thus natural to extend ADMM from 2 blocks to 𝑚 blocks, resulting in the iterative scheme: 𝑥₁^𝑘+1=argmin𝑓₁𝑥₁−𝜆^𝑘𝑇𝐴₁𝑥₁+𝛽2𝐴₁𝑥₁+𝐴₂𝑥^𝑘₂+⋅⋅⋅+𝐴_𝑚𝑥^𝑘_𝑚−𝑏²∣𝑥₁∈𝒳₁;𝑥₂^𝑘+1=argmin𝑓₂𝑥₂−𝜆^𝑘𝑇𝐴₂𝑥₂+𝛽2𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂+⋯+𝐴_𝑚𝑥^𝑘_𝑚−𝑏²∣𝑥₂∈𝒳₂;⋮𝑥_𝑚^𝑘+1=argmin𝑓_𝑚𝑥_𝑚−𝜆^𝑘𝑇𝐴_𝑚𝑥_𝑚+𝛽2𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1⋯+𝐴_𝑚𝑥_𝑚−𝑏²∣𝑥_𝑚∈𝒳_𝑚;𝜆^𝑘+1=𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+⋯+𝐴_𝑚𝑥_𝑚^𝑘+1−𝑏. (4)

Unfortunately, the convergence of the natural extension is still open under convex assumption, and the recent convergence results [7] are under the assumption that all the functions involved in the objective functions are strongly convex. This lack of convergence has inspired some ADM-based methods, for example, prediction-correction type method [3, 8–11], that is, the iterate 𝑥₁^𝑘+1,𝑥₂^𝑘+1,…,𝑥_𝑚^𝑘+1 is regarded as a prediction, and the next iterate is a correction for it. However, the numerical results show that the algorithm (4) always performs better than these variants. Recently, Han and Yuan [7] show that the global convergence of the extension of ADMM for 𝑚≥3 is valid if the involved functions are further assumed to be strongly convex. This result does not answer the open problem regarding the convergence of the extension of ADMM under the convex assumption, but it makes a key progress towards this objective.

In this paper, we consider the separable convex optimization problem (1) where each individual function 𝑓_𝑖 is the combination of a strongly convex function 𝑔_𝑖 and a linear transform 𝐵_𝑖. That is, (1) takes the following form: min^𝑚_𝑖=1𝑔_𝑖𝐵_𝑖𝑥_𝑖∣^𝑚_𝑖=1𝐴_𝑖𝑥_𝑖=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,2,…,𝑚, (5) where 𝑔_𝑖∶ℛ^𝑠𝑖→ℛ∪{+∞} (𝑖=1,2,…,𝑚) are closed proper strongly convex function with the modulus 𝜇_𝑖 (not necessarily smooth); 𝐴_𝑖∈ℛ^{𝑙×𝑛𝑖} (𝑖=1,2,…,𝑚); 𝒳_𝑖⊆ℛ^𝑛𝑖 (𝑖=1,2,…,𝑚) are closed convex sets; 𝑏∈ℛ^𝑙 and ∑^𝑚_𝑖=1𝑛_𝑖=𝑛; 𝐵_𝑖∈ℛ^{𝑠𝑖×𝑛𝑖} (𝑖=1,2,…,𝑚), where 𝐵_𝑖 may not have full column rank (if 𝐵_𝑖 has full column rank, the composite function is strongly convex and reduces to the case considered in [7]). Note that although (5) is very special, it arises frequently from many applications. One example is under the route-based traffic assignment problem [12], where 𝑔_𝑖 is the link traffic cost, 𝐵_𝑖 is the link-path incidence matrix, and 𝑥 is the path follow vector.

In the following, we abuse a little the notation and still write 𝑔_𝑖 with 𝑓_𝑖; that is, the problem under consideration is min^𝑚_𝑖=1𝑓_𝑖𝐵_𝑖𝑥_𝑖∣^𝑚_𝑖=1𝐴_𝑖𝑥_𝑖=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,2,…,𝑚, (6) where 𝑓_𝑖∶ℛ^𝑠𝑖→ℛ∪{+∞} (𝑖=1,2,…,𝑚) are closed proper strongly convex function with the modulus 𝜇_𝑖 (not necessarily smooth).

The rest of the paper is organized as follows. In the next section, we list some necessary preliminary results that will be used in the rest of the paper. We then describe the algorithm formally and analyze its global convergence under reasonable conditions in Section 3. We complete the paper with some conclusions in Section 4.

2. Preliminaries

In this section, we summarize some basic concepts and their properties that will be useful for further discussion.

Let ‖⋅‖_𝑝 denote the standard definition of the 𝑙^𝑝-norm, and particularly, let ‖⋅‖=‖⋅‖₂ denote the Euclidean norm. For a symmetric and positive definite matrix 𝐺, we denote ‖⋅‖_𝐺 the 𝐺-norm, that is, ‖𝑥‖_𝐺=𝑥^𝑇𝐺𝑥. If 𝐺 is the product of a positive parameter 𝛽 and the identity matrix 𝐼, that is, 𝐺=𝛽𝐼, we use the simpler notation: ‖⋅‖_𝐺=‖⋅‖_𝛽.

Let 𝑓∶ℛ^𝑛→ℛ∪{+∞}. If the domain of 𝑓 denoted by dom𝑓={𝑥∈ℛ^𝑛∣𝑓(𝑥)<+∞} is not empty, then 𝑓 is said to be proper. If for any 𝑥∈ℛ^𝑛 and 𝑦∈ℛ^𝑛, we have 𝑓𝑡𝑥+(1−𝑡)𝑦≤𝑡𝑓(𝑥)+(1−𝑡)𝑓𝑦,∀𝑡∈[0,1], (7) then 𝑓 is said to be convex. Furthermore, 𝑓 is said to be strongly convex with the modulus 𝜇>0 if and only if 𝑓()𝑦(𝑥)+()𝑓𝑦−1𝑡𝑥+1−𝑡≤𝑡𝑓1−𝑡2𝜇𝑡(1−𝑡)𝑥−𝑦²,∀𝑡∈[0,1]. (8)

A set-valued operator 𝑇 defined on ℛ^𝑛 is said to be monotone if and only if (𝑢−𝑢)^𝑇(𝑤−𝑤)≥0,∀𝑤∈𝑇𝑢,∀𝑤∈𝑇𝑢, (9) and 𝑇 is said to be strongly monotone with modulus 𝜇>0 if and only if (𝑢−𝑢)^𝑇(𝑤−𝑤)≥𝜇‖𝑢−𝑢‖²,∀𝑤∈𝑇𝑢,∀𝑤∈𝑇𝑢. (10)

Let Γ₀(ℛ^𝑛) denote the set of closed proper convex functions from ℛ^𝑛 to ℛ∪{+∞}. For any 𝑓∈Γ₀(ℛ^𝑛), the subdifferential of 𝑓 which is the set-valued operator, defined by 𝜕𝑓∶𝑥⟼𝜉∈ℛ^𝑛∣𝑦−𝑥^𝑇𝜉+𝑓(𝑥)≤𝑓𝑦,∀𝑦∈dom𝑓, (11) is monotone. Moreover, if 𝑓 is strongly convex function with the modulus 𝜇, 𝜕𝑓 is strongly monotone with the modulus 𝜇.

Let 𝐹 be a mapping from a set Ω⊂ℛ^𝑛→ℛ^𝑛. Then 𝐹 is said to be co-coercive on Ω with modulus 𝛾>0, if (𝑢−𝑣)^𝑇(𝐹(𝑢)−𝐹(𝑣))≥𝛾‖𝐹(𝑢)−𝐹(𝑣)‖²,∀𝑢,𝑣∈Ω. (12)

Throughout the paper, we make the following assumptions.

Assumption 1. (i) 𝑛_𝑖‖𝐵_𝑖𝑥_𝑖‖≥‖𝐴_𝑖‖‖𝑥_𝑖‖, ∀𝑥_𝑖∈ℛ^𝑛𝑖, 𝑖∈{1,2,…,𝑚}; (ii) the solution set of (1) is nonempty.

Remark 2. Assumption 1 is a little restrictive. However, some problems can satisfy it. A remarkable one is the following route-based traffic assignment problem.
Consider a transportation network 𝐺(𝒩,𝐸), where 𝒩 is the set of nodes. We denote the set of links by 𝒜, and the number of the element of 𝒜 by 𝑁_𝒜, respectively. Let RS denote the set of origin-destination (O-D) pairs. For an O-D pair rs∈RS, let 𝑞^rs be its traffic demand; let 𝑃^rs be the set of routes connecting rs, and 𝑝∈𝑃^rs; 𝒩^rs denotes the number of the routes connecting rs; let ℎ^rs_𝑝 be the route flow on 𝑝. The feasible route flow vector ℎ=(𝑝∈𝑃^rs∣rs∈RS) is thus given by 𝐻=ℎ∣_{𝑝∈𝑃rs}ℎ^rs_𝑝=𝑞^rs,ℎ^rs_𝑝≥0,∀𝑝∈𝑃^rs=,rs∈RSℎ∣𝑒^𝑇ℎ^rs₁,ℎ^rs₂,…,ℎ^rs_𝑁^rs=𝑞^rs,ℎ^rs_𝑝≥0,∀𝑝∈𝑃^rs,rs∈RS. (13) Define 𝐸 as the link-route incidence matrix such that 𝛿^𝑎_𝑝=1,if𝑝containslink𝑎0,otherwise. (14) Then, link flow 𝑓_𝑎 can be written as 𝑓_𝑎=_{rs∈RS𝑝∈𝑃rs}𝛿^𝑎_𝑝ℎ^rs_𝑝.,∀𝑎∈𝒜,𝐹=𝐸𝐻=𝑓∣𝑓=𝐸ℎ,ℎ∈𝐻 (15) By denoting the link cost function as 𝐶_𝑎(𝑓) and for the additive case, the route cost function as 𝐶_𝑝(ℎ), they can be related by 𝐶_𝑝ℎ=_𝑎∈𝒜𝛿^𝑎_𝑝𝐶_𝑎𝑓. (16) The user equilibrium traffic assignment problem can be formulated as a VI: find 𝑓^∗∈𝐹 such that 𝑓−𝑓^∗𝑇𝐶𝑓^∗≥0,∀𝑓∈𝐹, (17) or equivalently, find ℎ^∗∈𝐻 such that ℎ−ℎ^∗𝑇𝐸^𝑇𝐶𝐸ℎ^∗≥0,∀ℎ∈𝐻, (18) where 𝐶={𝐶_𝑎} is the vector of the link cost function.
In general, it is easy to show that 𝑒 is a row of 𝐸 and 𝐸 is not a full column rank (if 𝐸 is, then the above variational inequality is strongly monotone).
For simplicity, in the following, we only consider the case for 𝑚=3. Notice that for 𝑚≥3, it can be proved similarly following the processing of 𝑚=3.

3. The Method

In this section, we consider the following convex minimization problem with linear constraint, where the objective function is in the form of the sum of three individual functions without coupled variable: min𝑓₁𝐵₁𝑥₁+𝑓₂𝐵₂𝑥₂+𝑓₃𝐵₃𝑥₃s.t.𝐴₁𝑥₁+𝐴₂𝑥₂+𝐴₃𝑥₃=𝑏,𝑥_𝑖∈𝒳_𝑖,𝑖=1,2,3, (19) where 𝑓_𝑖∶ℛ^𝑠𝑖→ℛ∪{+∞} (𝑖=1,2,3) are closed proper strongly convex function with the modulus 𝜇_𝑖 (not necessarily smooth); 𝐵_𝑖∈ℛ^{𝑠𝑖×𝑛𝑖} (𝑖=1,2,3), 𝐴_𝑖∈ℛ^{𝑙×𝑛𝑖} (𝑖=1,2,3); 𝒳_𝑖⊆ℛ^𝑛𝑖 (𝑖=1,2,3) are closed convex sets; 𝑏∈ℛ^𝑙 and ∑³_𝑖=1𝑛_𝑖=𝑛.

The iterative scheme of ADMM for problem (19) is as follows: 𝑥₁^𝑘+1=argmin𝑓₁𝐵₁𝑥₁−𝜆^𝑘𝑇𝐴₁𝑥₁+𝛽2𝐴₁𝑥₁+𝐴₂𝑥^𝑘₂+𝐴₃𝑥^𝑘₃−𝑏²∣𝑥₁∈𝒳₁,𝑥₂^𝑘+1=argmin𝑓₂𝐵₂𝑥₂−𝜆^𝑘𝑇𝐴₂𝑥₂+𝛽2𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂+𝐴₃𝑥^𝑘₃−𝑏²∣𝑥₂∈𝒳₂,𝑥₃^𝑘+1=argmin𝑓₃𝐵₃𝑥₃−𝜆^𝑘𝑇𝐴₃𝑥₃+𝛽2𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥₃−𝑏²∣𝑥₃∈𝒳₃,𝜆^𝑘+1=𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥₃^𝑘+1−𝑏, (20) where 𝜆^𝑘 is the Lagrangian multiplier associated with the linear constraints and 𝛽>0 is the penalty parameter.

4. Convergence

In this section, we prove the convergence of the extended ADMM for problem (19). As the assumptions aforementioned, by invoking the first-order necessary and sufficient condition for convex programming, we easily see that the problem (19) under the condition is characterized by the following variational inequality (VI): find 𝑢^∗∈𝒰 and 𝜉^∗_𝑖∈𝜕𝑓_𝑖(𝐵_𝑖𝑥^∗_𝑖) such that 𝑢−𝑢^∗𝑇𝑄𝑢^∗≥0,∀𝑢∈𝒰, (21) where 𝑥𝑢∶=¹𝑥₂𝑥₃𝜆(𝑢)𝐵,𝑄∶=^𝑇1𝜉₁−𝐴^𝑇₁𝜆𝐵^𝑇₂𝜉₂−𝐴^𝑇₂𝜆𝐵^𝑇₃𝜉₃−𝐴^𝑇₃𝜆𝐴₁𝑥₁+𝐴₂𝑥₂+𝐴₃𝑥₃,−𝑏𝒰=𝒳₁×𝒳₂×𝒳₃×ℛ^𝑙. (22) We denote the VI (21)-(22) by MVI(𝒰,𝑄).

Similarly, in [7], we propose an easily implementable stopping criterion for executing (20): maxmax_1≤𝑖≤3𝐴_𝑖𝑥^𝑘_𝑖−𝐴_𝑖𝑥_𝑖^𝑘+1,³_𝑖=1𝐴_𝑖𝑥^𝑘_𝑖−𝑏≤𝜖, (23) and its rationale can be seen in the following lemma.

Lemma 3 (see [7]). If ∑³_𝑖=1𝐴_𝑖𝑥^𝑘_𝑖−𝑏=0 and 𝐴_𝑖𝑥^𝑘_𝑖=𝐴_𝑖𝑥_𝑖^𝑘+1 (𝑖=1,2,3), then (𝑥₁^𝑘+1,𝑥₂^𝑘+1,𝑥₃^𝑘+1,𝜆^𝑘+1) is a solution of MVI(𝒰,𝑄).

Lemma 3 implies that the iterate {(𝑥^𝑘₁,𝑥^𝑘₂,𝑥^𝑘₃,𝜆^𝑘)} is a solution of MVI(𝒰,𝑄) when the inequality (23) holds with 𝜖=0. Some techniques of establishing the error bounds in [13] can help us analyze how precisely the iterate satisfies the optimality conditions when the proposed stopping criterion is satisfied with a tolerance 𝜖>0.

Lemma 4. Let (𝑥^∗₁,𝑥^∗₂,𝑥^∗₃,𝜆^∗) be the solution of the problem (19), and let 𝜆^∗ be a corresponding Lagrange multiplier associated with the linear constraint. Then, the sequence {(𝑥^𝑘₁,𝑥^𝑘₂,𝑥^𝑘₃,𝜆^𝑘)} generated by (20) satisfies 𝜆^𝑘−𝜆^∗𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1≥−𝑏_3𝑖=1𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉_𝑖^𝑘+1−𝐵^𝑇_𝑖𝜉^∗_𝑖+𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²+𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇×𝐴₂𝑥^𝑘₂−𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1+𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1. (24)

Proof. By invoking the first-order optimality condition for the 𝑥_𝑖^𝑘+1-related subproblem in (20), for any 𝑥_𝑖∈𝒳_𝑖, 𝑖=1,2,3, we get 𝑥₁−𝑥₁^𝑘+1𝑇𝐵^𝑇₁𝜉₁^𝑘+1−𝐴^𝑇₁𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥^𝑘₂+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,₂−𝑥₂^𝑘+1𝑇𝐵^𝑇₂𝜉₂^𝑘+1−𝐴^𝑇₂×𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,₃−𝑥₃^𝑘+1𝑇𝐵^𝑇₃𝜉₃^𝑘+1−𝐴^𝑇₃×𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥₃^𝑘+1−𝑏≥0. (25) Setting 𝑥_𝑖=𝑥^∗_𝑖 (𝑖=1,2,3) in (25), we have 𝑥^∗₁−𝑥₁^𝑘+1𝑇𝐵^𝑇₁𝜉₁^𝑘+1−𝐴^𝑇₁𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥^𝑘₂+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,_∗2−𝑥₂^𝑘+1𝑇𝐵^𝑇₂𝜉₂^𝑘+1−𝐴^𝑇₂×𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,_∗3−𝑥₃^𝑘+1𝑇𝐵^𝑇₃𝜉₃^𝑘+1−𝐴^𝑇₃×𝜆^𝑘−𝛽𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥₃^𝑘+1−𝑏≥0. (26) On the other hand, setting (𝑥₁,𝑥₂,𝑥₃)=(𝑥₁^𝑘+1,𝑥₂^𝑘+1,𝑥₃^𝑘+1) in (21), it follows that 𝑥₁^𝑘+1−𝑥^∗₁𝑥₂^𝑘+1−𝑥^∗₂𝑥₃^𝑘+1−𝑥^∗₃^𝑇𝐵^𝑇₁𝜉^∗₁−𝐴^𝑇₁𝜆^∗𝐵^𝑇₂𝜉^∗₂−𝐴^𝑇₂𝜆^∗𝐵^𝑇₃𝜉^∗₃−𝐴^𝑇₃𝜆^∗≥0. (27) Adding (26) and (27), we obtain 𝑥₁^𝑘+1−𝑥^∗₁^𝑇𝐵^𝑇₁𝜉^∗₁−𝐵^𝑇₁𝜉₁^𝑘+1−𝐴^𝑇₁𝜆^∗−𝜆^𝑘−𝛽𝐴^𝑇₁𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥^𝑘₂+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,_2𝑘+1−𝑥^∗₂^𝑇𝐵^𝑇₂𝜉^∗₂−𝐵^𝑇₂𝜉₂^𝑘+1−𝐴^𝑇₂𝜆^∗−𝜆^𝑘−𝛽𝐴^𝑇₂𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃𝑥−𝑏≥0,_3𝑘+1−𝑥^∗₃^𝑇𝐵^𝑇₃𝜉^∗₃−𝐵^𝑇₃𝜉₃^𝑘+1−𝐴^𝑇₃𝜆^∗−𝜆^𝑘−𝛽𝐴^𝑇₃𝐴₁𝑥₁^𝑘+1+𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥₃^𝑘+1−𝑏≥0. (28) With the rearrangement of the above inequalities, we derive that 𝑥₁^𝑘+1−𝑥^∗₁^𝑇𝐴^𝑇₁𝜆^𝑘−𝜆^∗≥𝑥₁^𝑘+1−𝑥^∗₁^𝑇𝐵^𝑇₁𝜉₁^𝑘+1−𝐵^𝑇₁𝜉^∗₁+𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1𝐴−𝑏+𝛽₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇×𝐴₂𝑥^𝑘₂−𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1,𝑥₂^𝑘+1−𝑥^∗₂^𝑇𝐴^𝑇₂𝜆^𝑘−𝜆^∗≥𝑥₂^𝑘+1−𝑥^∗₂^𝑇𝐵^𝑇₂𝜉₂^𝑘+1−𝐵^𝑇₂𝜉^∗₂+𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1𝐴−𝑏+𝛽₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1,𝑥₃^𝑘+1−𝑥^∗₃^𝑇𝐴^𝑇₃𝜆^𝑘−𝜆^∗≥𝑥₃^𝑘+1−𝑥^∗₃^𝑇𝐵^𝑇₃𝜉₃^𝑘+1−𝐵^𝑇₃𝜉^∗₃+𝛽𝐴₃𝑥₃^𝑘+1−𝐴₃𝑥^∗₃^𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏. (29) Adding the above inequalities (29), we have 𝜆^𝑘−𝜆^∗𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1≥−𝑏_3𝑖=1𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉_𝑖^𝑘+1−𝐵^𝑇_𝑖𝜉^∗_𝑖+𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²+𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇𝐴₂𝑥^𝑘₂−𝐴₂𝑥₂^𝑘+1+𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1+𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1. (30) The proof is complete.

Hereafter, we define a matrix which will make the notation of proof more succinct. More specifically, let 𝑀=2𝛽𝐴^𝑇₁𝐴₁00002𝛽𝐴^𝑇₂𝐴₂0002𝛽𝐴^𝑇₃𝐴₃01000𝛽𝐼. (31) Obviously, 𝑀 is a positive semidefinite matrix, only for analysis convenience; we denote ‖𝑢‖²_𝑀=2𝛽𝐴₁𝑥₁²+𝐴₂𝑥₂²+𝐴₃𝑥₃²+‖𝜆‖²_1/𝛽. (32)

Lemma 5. Let 𝑢^∗=(𝑥^∗₁,𝑥^∗₂,𝑥^∗₃,𝜆^∗) be a solution of MVI(𝒰,𝑄), and let the sequence {(𝑥^𝑘₁,𝑥^𝑘₂,𝑥^𝑘₃,𝜆^𝑘)} be generated by (20). Then, one has 𝑢^𝑘+1−𝑢^∗2_𝑀≤𝑢^𝑘−𝑢^∗2_𝑀+³_𝑖=13𝛽𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²−2³_𝑖=1𝜇_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖². (33)

Proof. From (20) and Lemma 4, we have 𝜆^𝑘+1−𝜆^∗2_1/𝛽=𝜆^𝑘−𝜆^∗−𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²_1/𝛽=𝜆^𝑘−𝜆^∗2_1/𝛽−2𝜆^𝑘−𝜆^∗𝑇3_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏+𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²≤𝜆^𝑘−𝜆^∗2_1/𝛽−2³_𝑖=1𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉_𝑖^𝑘+1−𝐵^𝑇_𝑖𝜉^∗_𝑖−𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²−2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇׳_𝑖=2𝐴_𝑖𝑥^𝑘_𝑖−𝐴_𝑖𝑥_𝑖^𝑘+1−2𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1. (34) Since ³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²=³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖²+_𝑖≠𝑗𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐴_𝑗𝑥_𝑗^𝑘+1−𝑥^∗_𝑗, (35) and 𝐴₁𝑥^∗₁+𝐴₂𝑥^∗₂+𝐴₃𝑥^∗₃=𝑏, we can get −𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²=−𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖²−𝛽_𝑖≠𝑗𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐴_𝑗𝑥_𝑗^𝑘+1−𝑥^∗_𝑗. (36) Using Cauchy-Schwarz inequality, we have −2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇3_𝑖=2𝐴_𝑖𝑥^𝑘_𝑖−𝐴_𝑖𝑥_𝑖^𝑘+1−2𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥₃^𝑘+1=−2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇𝐴₂𝑥^𝑘₂−𝐴₂𝑥^∗₂+2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂−2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥^∗₃+2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁^𝑇𝐴₃𝑥₃^𝑘+1−𝐴₃𝑥^∗₃−2𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥^𝑘₃−𝐴₃𝑥^∗₃+2𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂^𝑇𝐴₃𝑥₃^𝑘+1−𝐴₃𝑥^∗₃≤2𝛽𝐴₁𝑥₁^𝑘+1−𝐴₁𝑥^∗₁²+𝛽𝐴₂𝑥₂^𝑘+1−𝐴₂𝑥^∗₂²+𝛽𝐴₂𝑥^𝑘₂−𝑥^∗₂²+2𝛽𝐴₃𝑥^𝑘₃−𝑥^∗₃²+𝛽_𝑖≠𝑗𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐴_𝑗𝑥_𝑗^𝑘+1−𝑥^∗_𝑗. (37) Substituting (36) and (37) into (34), we get 𝜆^𝑘+1−𝜆^∗2_1/𝛽≤𝜆^𝑘−𝜆^∗2_1/𝛽+2𝛽³_𝑖=1𝐴_𝑖𝑥^𝑘_𝑖−𝑥^∗_𝑖²+𝛽³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑥^∗_𝑖²−2³_𝑖=1𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉_𝑖^𝑘+1−𝐵^𝑇_𝑖𝜉^∗_𝑖. (38)
Since 𝑓_𝑖 is strongly convex, from the strong monotonicity of the subdifferential mapping 𝜕𝑓_𝑖 (with the modulus 𝜇_𝑖), then we have 𝑥_𝑖^𝑘+1−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉_𝑖^𝑘+1−𝐵^𝑇_𝑖𝜉^∗_𝑖=𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖^𝑇𝜉_𝑖^𝑘+1−𝜉^∗_𝑖≥𝜇_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖², (39) where 𝜉^∗_𝑖∈𝜕𝑓_𝑖(𝐵_𝑖𝑥^∗_𝑖), 𝜉_𝑖^𝑘+1∈𝜕𝑓_𝑖(𝐵_𝑖𝑥_𝑖^𝑘+1), for any 𝑖∈{1,2,3}.
By using the notion of ‖𝑢^𝑘+1−𝑢^∗‖²_𝑀, from (38) we have 𝑢^𝑘+1−𝑢^∗2_𝑀=𝜆^𝑘+1−𝜆^∗2_1/𝛽+2𝛽³_𝑖=1𝐴_𝑖(𝑥_𝑖^𝑘+1−𝑥^∗_𝑖)²≤𝜆^𝑘−𝜆^∗2_1/𝛽+2𝛽³_𝑖=1𝐴_𝑖(𝑥^𝑘_𝑖−𝑥^∗_𝑖)²+3𝛽³_𝑖=1𝐴_𝑖(𝑥_𝑖^𝑘+1−𝑥^∗_𝑖)²−2³_𝑖=1𝜇_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖²≤𝑢^𝑘−𝑢^∗2_𝑀+³_𝑖=13𝛽𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²−2³_𝑖=1𝜇_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖². (40) The proof is complete.

Theorem 6. Under Assumption 1, for any 0<𝛽<min_1≤𝑖≤32𝜇_𝑖3𝑛²_𝑖, (41) the sequence {(𝑥^𝑘₁,𝑥^𝑘₂,𝑥^𝑘₃,𝜆^𝑘)} generated by (20) converges to a solution of MVI(𝒰,𝑄).

Proof. From Lemma 5, we have 𝑢^𝑘+1−𝑢^∗2_𝑀≤𝑢^𝑘−𝑢^∗2_𝑀+³_𝑖=13𝛽𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²−2³_𝑖=1𝜇_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖², (42) where 0<𝛽<min_1≤𝑖≤32𝜇_𝑖3𝑛²_𝑖. (43) From Assumption 1, it follows that 𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²≤𝐴_𝑖²𝑥_𝑖^𝑘+1−𝑥^∗_𝑖²≤𝑛²_𝑖𝐵_𝑖𝑥_𝑖^𝑘+1−𝐵_𝑖𝑥^∗_𝑖²,𝑖=1,2,3. (44) Consequently, 𝑢^𝑘+1−𝑢^∗2_𝑀≤𝑢^𝑘−𝑢^∗2_𝑀+³_𝑖=13𝛽−2𝜇_𝑖𝑛²_𝑖𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖². (45) From (45), we have 𝑢^𝑘+1−𝑢^∗2_𝑀≤𝑢^𝑘−𝑢^∗2_𝑀≤⋯≤𝑢⁰−𝑢^∗2_𝑀<+∞, (46) which means that the generated sequence {𝑢^𝑘} is bounded.
Furthermore, it follows that ^+∞_𝑘=0³_𝑖=12𝜇_𝑖𝑛²_𝑖−3𝛽𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²≤^+∞_𝑘=0𝑢^𝑘−𝑢^∗_𝑀−𝑢^𝑘+1−𝑢^∗_𝑀<+∞, (47) which means that lim³_{𝑘→+∞𝑖=1}𝐴_𝑖𝑥_𝑖^𝑘+1−𝐴_𝑖𝑥^∗_𝑖²=0. (48) Therefore, we have lim_𝑘→+∞³_𝑖=1𝐴_𝑖𝑥_𝑖^𝑘+1−𝑏²=0. (49) Since ‖𝐴_𝑖‖ is nonzero and bounded, from (48) we have lim_𝑘→+∞𝑥_𝑖^𝑘+1−𝑥^∗_𝑖=0,∀𝑖=1,2,3. (50) Since {𝑢^𝑘} is bounded, {𝜆^𝑘} has at least one cluster point, say 𝜆. Let {𝜆^𝑘𝑗} be the corresponding subsequence that converges to 𝜆. Taking a limit along this subsequence in (25) and (49), we obtain 𝜉^∗_𝑖∈𝜕𝑓_𝑖(𝐵_𝑖𝑥^∗_𝑖), 𝑥_𝑖−𝑥^∗_𝑖^𝑇𝐵^𝑇_𝑖𝜉^∗_𝑖−𝐴^𝑇_𝑖𝜆≥0,∀𝑥_𝑖∈𝒳_𝑖,𝑖=1,2,3,_3𝑖=1𝐴_𝑖𝑥^∗_𝑖−𝑏=0, (51) which follows that 𝜆 is an optimal Lagrange multiplier. Since 𝜆^∗ is arbitrary, we can set 𝜆^∗=𝜆 in (46) and conclude that the whole generated sequence converges to a solution of MVI(𝒰,𝑄).

5. Conclusions

In this paper, we extend the convergence analysis of the ADMM for the separable convex optimization problem with strongly convex functions to the case in which the individual functions are composites of strongly convex functions with a linear transform. Under further assumptions, we established the global convergence of the algorithm.

It should be admitted that although some problems arising from applications such as traffic assignment fall into our analysis, the problems considered here are too special. Thus, it is far away to solve the open problem of convergence of the ADMM with more than three blocks.

Acknowledgments

Xingju Cai was supported by the National Natural Science Foundation of China (NSFC) Grants nos. 11071122 and 11171159 and by the Doctoral Fund of Ministry of Education of China no. 20103207110002.