Mathematical Problems in Engineering

Mathematical Problems in Engineering / 2012 / Article

Research Article | Open Access

Volume 2012 |Article ID 819607 |

Aiqun Huang, Chengxian Xu, "A Globally Convergent Filter-Type Trust Region Method for Semidefinite Programming", Mathematical Problems in Engineering, vol. 2012, Article ID 819607, 13 pages, 2012.

A Globally Convergent Filter-Type Trust Region Method for Semidefinite Programming

Academic Editor: Soohee Han
Received17 May 2012
Accepted29 Jun 2012
Published26 Aug 2012


When using interior methods for solving semidefinite programming (SDP), one needs to solve a system of linear equations at each iteration. For problems of large size, solving the system of linear equations can be very expensive. In this paper, based on a semismooth equation reformulation using Fischer's function, we propose a filter method with trust region for solving large-scale SDP problems. At each iteration we perform a number of conjugate gradient iterations, but do not need to solve a system of linear equations. Under mild assumptions, the convergence of this algorithm is established. Numerical examples are given to illustrate the convergence results obtained.

1. Introduction

Semidefinite programming (SDP) is convex programming over positive semidefinite matrices. For early application, SDP has been widely used in control theory and combinatorial optimization (see, e.g., [1–3]). Since some algorithms for linear optimization can be extended to many general SDP problems, that aroused much interest in SDP. In the past decade, many algorithms have been proposed for solving SDP, including interior-point methods (IPMs) [4–7], augmented methods [8–10], new Newton-type methods [11], modified barrier methods [12], and regularization approaches [13].

For small and medium sized SDP problems, IPMs are generally efficient. But for large-scale SDP problems, IPMs become very slow. In order to improve this shortcoming, [9, 14] proposed inexact IPMs using an iterative solver to compute a search direction at each iteration. More recently, [13] applied regularization approaches to solve SDP problems. All of these methods are first-order based on a gradient, or inexact second-order based on an approximation of Hessian matrix methods [15].

In this paper, we will extend filter-trust-region methods for solving linear (or nonlinear) programming [16] to large-scale SDP problems and use Lipschitz continuity. Furthermore, the accuracy of this method is controlled by a forcing parameter. It is shown that, under mild assumptions, this algorithm is convergent.

The paper is organized as follows. Some preliminaries are introduced in Section 2. In Section 3, we propose a filter-trust-region method for solving SDP problems, and we study the convergence of this method in Section 4. In Section 5, some numerical examples are presented to demonstrate the convergence results obtained in this paper. Finally, we give some conclusions in Section 6.

In this paper, we use the following common notation for SDP problems: 𝒳𝑛 and ℛ𝑚 denote the space of 𝑛×𝑛 real symmetric matrices and the space of vectors with 𝑚 dimensions, respectively; 𝑋≽0(𝑋≻0) denotes that 𝑋∈𝒳𝑛 is positive semidefinite (positive definite), and 𝑋⪯0(𝑋≺0) is used to indicate that 𝑋∈𝒳𝑛 is negative semidefinite (negative definite). A superscript 𝑇 represents transposes of matrices or vectors. For 𝑋,𝑌∈𝒳𝑛, the standard scalar product on the space of 𝒳𝑛 is defined by ⟨𝑋,𝑌⟩∶=𝑋•𝑌=trace(𝑋𝑌)=𝑛𝑖,𝑗=1𝑋𝑖,𝑗𝑌𝑖,𝑗.(1.1) If 𝑋∈𝒳𝑛 and 𝑥∈ℛ𝑚, we denote that ‖𝑋‖𝐹 is the Frobenius norm of 𝑋, that is, ‖𝑋‖𝐹=√⟨𝑋,𝑋⟩=∑𝑛𝑖,𝑗=1𝑋2𝑖,𝑗 and ‖𝑥‖2 is the 2-norm of 𝑥, that is, ‖𝑥‖2=√𝑥𝑇𝑥=∑𝑚𝑖=1𝑥2𝑖, respectively. Let 𝑋 be a ğ‘Ã—ğ‘ž matrix. Then we denote by Vec(𝑋) a ğ‘ğ‘ž vector made of columns of 𝑋 stacked one by one, and the operator Mat(⋅) is the inverse of Vec(⋅), that is, Mat(Vec(𝑋))=𝑋. We also denote that 𝐼 is identity matrix.

2. Preliminaries

We consider a SDP problem of the form min𝐶•𝑋subjectto𝒜(𝑋)=𝑏,𝑋⪰0,(2.1) where 𝐶∈𝒳𝑛, 𝐴(𝑖)∈𝒳𝑛, 𝑖=1,2,…,𝑚, and 𝑏=(𝑏1,𝑏2,…,𝑏𝑚)𝑇∈ℛ𝑚 are given dates; 𝒜 is a linear map from 𝒳𝑛 to ℛ𝑚 given by âŽ¡âŽ¢âŽ¢âŽ¢âŽ¢âŽ¢âŽ¢âŽ£ğ´ğ’œ(𝑋)∶=(1)𝐴•𝑋(2)⋮𝐴•𝑋(𝑚)âŽ¤âŽ¥âŽ¥âŽ¥âŽ¥âŽ¥âŽ¥âŽ¦â€¢ğ‘‹,𝑋∈𝒳𝑛.(2.2) The dual to the problem (2.1) is given by max𝑏𝑇𝑦subjectto𝒜∗(𝑦)+𝑆=𝐶,𝑆⪰0,(2.3) where 𝒜∗ is an adjoint operator of 𝒜∶ℛ𝑚→𝒳𝑛 given by 𝒜∗(𝑦)=𝑚𝑖=1𝑦𝑖𝐴(𝑖),𝑦∈ℛ𝑚.(2.4) Obviously, 𝑋∈𝒳𝑛 and (𝑦,𝑆)∈ℛ𝑚×𝒳𝑛 are the primal and dual variables, respectively.

It is easily verified that the SDP problem (2.1) is convex. When (2.1) and (2.3) have strictly feasible points, then strong duality holds, see [5, 12]. In this case, a point (𝑋,𝑦,𝑆) is optimal for SDP problems (2.1) and (2.3) if and only if 𝒜(𝑋)=𝑏,𝒜∗(𝑦)+𝑆=𝐶,𝑋⪰0,𝑆⪰0,⟨𝑋,𝑆⟩=0.(2.5) In the sense that (𝑋,𝑦,𝑆) solves SDP problems (2.1) and (2.3) if and only if (𝑋,𝑦,𝑆) solves (2.5) when both SDP problems (2.1) and (2.3) have strictly feasible points.

We now introduce some lemmas which will be used in the sequel.

Lemma 2.1 (see [17]). Let 𝐴,𝐵∈𝒳𝑛 and let 𝐴≽0,𝐵≽0. Then ⟨𝐴,𝐵⟩=0 if and only if 𝐴𝐵=0.

For 𝑋,𝑆∈𝒳𝑛, we define a mapping 𝜙∶𝒳𝑛×𝒳𝑛→𝒳𝑛 given by √𝜙(𝑋,𝑆)∶=𝑋+𝑆−𝑋2+𝑆2,(2.6) which is attributed by Fischer to Burmeister (see [18, 19]). This function is nondifferentiable and has a basic property.

Lemma 2.2 (see [20, Lemma 6.1]). Let 𝜙 be the Fischer-Burmeister function defined in (2.6). Then 𝜙(𝑋,𝑆)=0⟺𝑋⪰0,𝑆⪰0,𝑋𝑆=0.(2.7)

In addition, for 𝜏>0 and 𝑋,𝑆∈𝒳𝑛, we define a mapping 𝜙𝜏∶𝒳𝑛×𝒳𝑛→𝒳𝑛 by 𝜙𝜏√(𝑋,𝑆)∶=𝑋+𝑆−𝑋2+𝑆2+2𝜏2𝐼,(2.8) which is differentiable and has following results.

Lemma 2.3 (see [11, Proposition  2.3]). Let 𝜏>0 be any positive number and let 𝜙𝜏 be defined by (2.8). Then 𝜙𝜏(𝑋,𝑆)=0⟺𝑋≻0,𝑆≻0,𝑋𝑆=𝜏2𝐼.(2.9)

Lemma 2.4. Let 𝜏>0 be any positive number, and let 𝜙𝜏 be defined by (2.8). If 𝜏→0, we would have 𝜙𝜏(𝑋,𝑆)=0⟺𝑋⪰0,𝑆⪰0,𝑋𝑆=0.(2.10)

Proof. The proof can be obtained from Lemmas 2.2 and 2.3.

Lemma 2.5 (see [20, pages 170–171]). For any 𝐶≻0, define the linear operator 𝐿𝐶 by 𝐿𝐶[𝑋]∶=𝐶𝑋+𝑋𝐶,𝑋∈𝒳𝑛.(2.11) Then 𝐿𝐶 is strictly monotone and so has an inverse 𝐿𝐶−1.

Lemma 2.6 (see [21, Lemma  2]). Let 𝑋,𝑆,𝑈,𝑉∈𝒳𝑛, and let 𝜙𝜏 be defined by (2.8). For any 𝜏>0, we have that 𝜙𝜏 is Fréchet-differentiable and ∇𝜙𝜏(𝑋,𝑆)(𝑈,𝑉)=𝑈+𝑉−𝐿𝐶−1[],𝑋𝑈+𝑈𝑋+𝑆𝑉+𝑉𝑆(2.12) where √𝐶∶=𝑋2+𝑆2+2𝜏2𝐼.

Lemma 2.7 (see [22, Corollary 2.7]). Let 𝐹 be a map from 𝒳𝑛 to 𝒳𝑛. If 𝐹 is locally Lipschitzian on 𝒳𝑛, then 𝐹 is almost everywhere Fréchet-differentiable on 𝒳𝑛.

3. The Algorithm

In this section, we will present a filter-trust-region method for solving SDP problems (2.1) and (2.3). Firstly, for a parameter 𝜏>0, we construct a function: ğ»ğœâŽ›âŽœâŽœâŽœâŽœâŽœâŽœâŽğœğ’œ(𝑋,𝑦,𝑆)∶=𝒜(𝑋)−𝑏∗√(𝑦)+𝑆−𝐶𝑋+𝑆−𝑋2+𝑆2+2𝜏2ğ¼âŽžâŽŸâŽŸâŽŸâŽŸâŽŸâŽŸâŽ ,(3.1) where (𝑋,𝑦,𝑆)∈𝒳𝑛×ℛ𝑚×𝒳𝑛.

According to Lemmas 2.1, 2.3 and 2.4, the following theorem is obvious.

Theorem 3.1. Let 𝜏>0 and let 𝐻𝜏(𝑋,𝑦,𝑆) be defined by (3.1). If SDP problems (2.1) and (2.3) have strictly feasible points, then 𝐻𝜏𝑋∗,𝑦∗,𝑆∗𝑋=0⟹∗,𝑦∗,𝑆∗solves(2.5).(3.2)

In what follows, we will study properties of the function 𝐻𝜏(𝑋,𝑦,𝑆). For simplicity, in the remaining sections of this paper, we denote 𝑍∶=(𝑋,𝑦,𝑆), 𝑍𝑘∶=(𝑋𝑘,𝑦𝑘,𝑆𝑘) and Δ𝑍∶=(Δ𝑋,Δ𝑦,Δ𝑆).

Theorem 3.2. Let 𝐻𝜏(𝑍) be defined by (3.1). For any 𝑍,Δ𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛 and 𝜏>0, then 𝐻𝜏(𝑍) is Fréchet-differentiable and âˆ‡ğ»ğœâŽ›âŽœâŽœâŽœâŽœâŽœâŽœâŽğ’œ(𝑍)(Δ𝑍)=Δ𝜏𝒜(Δ𝑋)−𝑏∗(Δ𝑦)+Δ𝑆−𝐶Δ𝑋+Δ𝑆−𝐿𝐶−1[]âŽžâŽŸâŽŸâŽŸâŽŸâŽŸâŽŸâŽ ğ‘‹Î”ğ‘‹+Δ𝑋𝑋+𝑆Δ𝑆+Δ𝑆𝑆,(3.3) where Δ𝜏>0 and √𝐶∶=𝑋2+𝑆2+2𝜏2𝐼.

Proof. For any 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛, since 𝒜(𝑋)−𝑏 and 𝒜∗(𝑦)+𝑆−𝐶 are linear functions and continuous differentiable, it follows that they are also locally Lipschitz continuous. Then, from Lemma 2.7, 𝒜(𝑋)−𝑏 and 𝒜∗(𝑦)+𝑆−𝐶 are Fréchet-differentiable. Furthermore, √𝑋+𝑆−𝑋2+𝑆2+2𝜏2𝐼 is Fréchet-differentiable from Lemma 2.6. Thus, 𝐻𝜏(𝑍) is Fréchet-differentiable and has the form of (3.3). We complete the proof.

We endow the variable 𝑍 with the following norm: ‖‖𝑍‖=‖(𝑋,𝑦,𝑆)‖∶=𝑋‖2𝐹+‖𝑦‖22+‖𝑆‖2𝐹1/2.(3.4) In addition, we set ℎℎ(𝑍)=1(𝑍),ℎ2(𝑍),ℎ3(𝑍),ℎ4(𝑍)𝑇,(3.5) where ℎ1(𝑍)=‖𝒜(𝑋)−𝑏‖2,ℎ2‖‖𝒜(𝑍)=∗‖‖(𝑦)+𝑆−𝐶𝐹,ℎ3‖‖‖√(𝑍)=𝑋+𝑆−𝑋2+𝑆2+2𝜏2ğ¼â€–â€–â€–ğ¹â„Ž4(𝑍)=|𝜏|.(3.6) We also define the function 𝐻𝜏(𝑍) and the vector ℎ(𝑍) with the following norm: ‖‖𝐻𝜏‖‖(𝑍)=‖ℎ(𝑍)‖=4𝑖=1â„Žğ‘–(𝑍)21/2=‖‖𝒜(𝑋)−𝑏22+‖‖𝒜∗‖‖(𝑦)+𝑆−𝐶2𝐹+‖‖‖√𝑋+𝑆−𝑋2+𝑆2+2𝜏2𝐼‖‖‖2𝐹+𝜏21/2.(3.7)

Now, for any 𝜏>0, we define the merit function Ψ∶𝒳𝑛×ℛ𝑚×𝒳𝑛→ℛ by Ψ𝜏1(𝑍)∶=2‖‖𝐻𝜏‖‖(𝑍)2.(3.8)

Lemma 3.3. For any 𝜏>0 and 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛, if 𝑋 and 𝑆 are nonsingular, then Ψ𝜏(𝑍) is locally Lipschitz continuous and twice Fréchet-differentiable at every 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛.

Proof. For any 𝜏>0, since Ψ𝜏(𝑍) is convex and continuously differentiable, it follows that Ψ𝜏(𝑍) is also locally Lipschitz continuous.
In addition, for any 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛, from [20, pages 173–175], ℎ3(𝑍)2 is twice Fréchet-differentiable. Furthermore, ℎ1(𝑍)2, ℎ2(𝑍)2, and ℎ4(𝑍)2 are continuous at every 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛 when 𝜏>0, which, together with Lemma 2.7, Ψ𝜏(𝑍) is twice Fréchet-differentiable. The proof is completed.

Lemma 3.4. Let 𝐻𝜏(𝑍) and Ψ𝜏(𝑍) be defined by (3.1) and (3.8), respectively. For any 𝜏>0, we have Ψ𝜏(𝑍)=0⟺𝐻𝜏(𝑍)=0.(3.9)

Proof. The proof can be immediately obtained from the definition of 𝐻𝜏(𝑍) and Ψ𝜏(𝑍).

We follow the classical method for solving Ψ𝜏(𝑍)=0, which consists some norm of the residual. For any 𝜏>0, we consider minΨ𝜏(𝑍),(3.10) where 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛. Thus, for any 𝜏>0, we want to find a minimizer 𝑍∗ of Ψ𝜏(𝑍). Furthermore, if Ψ𝜏(𝑍∗)=0, then 𝑍∗ is also a solution of 𝐻𝜏(𝑍).

In order to state our method for solving (3.10), we consider using a filter mechanism to accept a new point. Just as [16, pages 19–20], the notation of filter is based on that of dominance.

Definition 3.5. For any 𝜏>0 and any 𝑍1,𝑍2∈𝒳𝑛×ℛ𝑚×𝒳𝑛, a point 𝑍1 dominates a point 𝑍2 if and only if â„Žğ‘–î€·ğ‘1î€¸â‰¤â„Žğ‘–î€·ğ‘2∀𝑖=1,2,3,4.(3.11)

Thus, if iterate 𝑍1 dominates iterate 𝑍2, the latter is of no real interest to us since 𝑍1 is at least as good as 𝑍2 for each of the components of ℎ(𝑍). All we need to do is remember iterates that are no dominated by other iterates by using a structure called a filter.

Definition 3.6. Let 𝐹(𝑘) be a set of 4-tuples of the following form: ℎ1𝑍𝑘,ℎ2𝑍𝑘,ℎ3𝑍𝑘,ℎ4𝑍𝑘.(3.12) We define 𝐹(𝑘) as a filter if ℎ(𝑍𝑘) and ℎ(𝑍𝑙) belong to 𝐹(𝑘), when 𝑘≠𝑙, then â„Žğ‘–î€·ğ‘ğ‘˜î€¸<â„Žğ‘–î€·ğ‘ğ‘™î€¸foratleastone𝑖∈{1,2,3,4}.(3.13)

Definition 3.7. A new point 𝑍+𝑘 is acceptable for the filter 𝐹(𝑘) if and only if î€·ğ‘âˆ€â„Žğ‘˜î€¸âˆˆğ¹(𝑘)∃𝑖∈{1,2,3,4}âˆ¶â„Žğ‘–î€·ğ‘+ğ‘˜î€¸â‰¤â„Žğ‘–î€·ğ‘ğ‘˜î€¸â€–â€–â„Žî€·ğ‘âˆ’ğ›¼ğ‘˜î€¸â€–â€–,(3.14) where √𝛼∈(0,1/4) is a small constant.

Now, we formally present our trust region algorithm by using filter techniques.

Algorithm 3.8. The Filter-Trust-Region Algorithm
Step 0. Choose an initial point 𝑍0=(𝑋0,𝑦0,𝑆0)∈𝒳𝑛×ℛ𝑚×𝒳𝑛, 𝜀>0, √0<𝛼<1/4 and 𝜏0=⟨𝑋0,𝑆0⟩/2𝑛. The constants 𝜂1, 𝜂2, 𝜂3, 𝜇, 𝛾, 𝛾1, and 𝛾2 are also given and satisfy 0<𝜂1≤𝜂2≤𝜂3<1,0<𝜇<1,0<𝛾<𝛾1<1≤𝛾2.(3.15)
Compute Ψ𝜏0(𝑍0), set Δ0=0.5‖∇Ψ𝜏0(𝑍0)‖, 𝑘=0 and only (𝜇,−∞,𝜇,𝜇) in the filter 𝐹(0).
Step 1. If ∇Ψ𝜏𝑘(𝑍𝑘)<𝜀, stop.
Step 2. Compute Δ𝑍𝑘 by solving the following problem: min𝜑𝑘(Δ𝑍)s.t.‖Δ𝑍‖≤Δ𝑘,(3.16) where 𝜑𝑘1(Δ𝑍)=2‖‖𝐻𝜏𝑘𝑍𝑘+∇𝐻𝜏𝑘𝑍𝑘‖‖(Δ𝑍)2=Ψ𝜏𝑘𝑍𝑘+𝐻𝜏𝑘𝑍𝑘𝑇∇𝐻𝜏𝑘𝑍𝑘(+1Δ𝑍)2(Δ𝑍)𝑇∇𝐻𝜏𝑘𝑍𝑘𝑇∇𝐻𝜏𝑘𝑍𝑘(Δ𝑍).(3.17)
If ‖Δ𝑍𝑘‖<𝜀, stop.
Otherwise, computer the trial point 𝑍+𝑘=𝑍𝑘+Δ𝑍𝑘.
Step 3. Compute Ψ𝜏𝑘(𝑍+𝑘) and define the following ratio: 𝑟𝑘=Ψ𝜏𝑘𝑍𝑘−Ψ𝜏𝑘𝑍+𝑘𝜑𝑘(0)−𝜑𝑘Δ𝑍𝑘.(3.18)
Step 4. If 𝑟𝑘≥𝜂1, set 𝑍𝑘+1=𝑍+𝑘.
If 𝑟𝑘<𝜂1 but 𝑍+𝑘 satisfies (3.14), then add ℎ(𝑍+𝑘) to the filter 𝐹(𝑘) and remove all points from 𝐹(𝑘) dominated by ℎ(𝑍+𝑘). At the same time, set 𝑍𝑘+1=𝑍+𝑘.
Else, set 𝑍𝑘+1=𝑍𝑘.
Step 5. Update 𝜏𝑘 by choosing 𝜏𝑘+1∈𝛾𝜏𝑘if𝑍𝑘+1=𝑍+𝑘,𝜏𝑘else;(3.19) and update trust-region radius Δ𝑘 by choosing Δ𝑘+1⎧⎪⎨⎪⎩∶=𝛾Δ𝑘,if𝑟𝑘<𝜂1,𝛾1Δ𝑘,if𝑟𝑘∈𝜂1,𝜂2,Δ𝑘,if𝑟𝑘∈𝜂2,𝜂3,𝛾2Δ𝑘,if𝑟𝑘≥𝜂3.(3.20)
Step 6. Set 𝑘∶=𝑘+1 and go to Step 1.

Remark 3.9. Algorithm 3.8 can be started any 𝜏>0. In fact, in order to increase the convergent speed greatly, we always choose 𝜏0=⟨𝑋0,𝑆0⟩/2𝑛. In addition, in this algorithm, we fix 𝜏 at first, then search 𝑍 for Ψ𝜏(𝑍)=0 to update 𝑍. At last we update 𝜏 and repeat.

The following lemma is a generalized case of Proposition 3.1 in [23].

Lemma 3.10. Algorithm 3.8 is well defined, that is, the inner iteration (Step 2) terminates finitely.

For the purpose of our analysis, in the sequence of points generated by Algorithm 3.8, we denote 𝒜={𝑘∣𝑟𝑘≥𝜂1},ℬ={𝑘|ℎ(𝑍+𝑘)isaddedtothefilter𝐹(𝑘)}, and ğ’ž={𝑘|𝑍𝑘+1=𝑍𝑘+Δ𝑍𝑘}. It is clear that, â‹ƒâ„¬ğ’ž=𝒜.

Remark 3.11. Lemma 3.3 implies that there exists a constant 0<𝑀≤1 such that â„Žğ‘–î€·ğ‘ğ‘˜î€¸â€–â€–âˆ‡â‰¤ğ‘€,2â„Žğ‘–î€·ğ‘ğ‘˜î€¸â€–â€–â€–â€–âˆ‡â‰¤ğ‘€,2𝜑𝑘‖‖(Δ𝑍)≤𝑀(3.21) for all ğ‘˜âˆˆğ’ž and 𝑖∈{1,2,3,4}. The second of above inequalities ensures that the constant 0<𝑀≤1 can also be chosen such that ‖‖∇2Ψ𝜏𝑘𝑍𝑘‖‖≤𝑀.(3.22)

4. Convergence of Analysis

In this section, we present a proof of global convergence of Algorithm 3.8. First, we make the following assumptions.

Some lemmas will be presented to be used in the subsequent analysis. (S1)𝜑𝑘(0)−𝜑𝑘(Δ𝑍𝑘)≥1/2‖∇Ψ𝜏𝑘(𝑍𝑘)‖min{Δ𝑘,‖∇Ψ𝜏𝑘(𝑍𝑘)‖/‖∇𝐻𝜏𝑘(𝑍𝑘)𝑇∇𝐻𝜏𝑘(𝑍𝑘)‖}, where Δ𝑍𝑘 is a solution of (3.16). (S2)  The iterations generated by Algorithm 3.8 remain in a close, bounded domain.

Lemma 4.1 (see [24]). Let assumptions (S1) and (S2) hold. If there exists 𝑙0>0 such that ‖∇Ψ𝜏𝑘(𝑍𝑘)‖≥𝑙0>0 for all 𝑘; then there exists 𝑙1>0 such that Δ𝑘≥𝑙1.

Lemma 4.2. Let {𝜏𝑘} be the infinite sequence generated by the Algorithm 3.8. Then limğ‘˜â†’âˆžğœğ‘˜=0.(4.1)

Proof. Since |ğ’ž|=|𝒜|=+∞, from Steps 4 and 5 of Algorithm 3.8, 𝜏𝑘+1=𝛾𝜏𝑘 and 0<𝛾<𝜏0<1. Therefore, 𝜏𝑘+1=𝛾𝑘𝜏0. Moreover, limğ‘˜â†’âˆžğœğ‘˜=limğ‘˜â†’âˆžğ›¾ğ‘˜ğœ0=0(4.2) for 0<𝛾<𝜏0<1, which completes the proof.

Theorem 4.3. Let |ğ’ž|<+∞, assumptions (S1) and (S2) hold. Then there exists ğ‘˜âˆˆğ’ž such that ∇Ψ𝜏𝑘𝑍𝑘=0.(4.3)

Proof. Suppose that ∇Ψ𝜏𝑘(𝑍𝑘)≠0 for all ğ‘˜âˆˆğ’ž. Then there exists 𝜔0>0 such that ‖‖∇Ψ𝜏𝑘𝑍𝑘‖‖≥𝜔0>0.(4.4) From Lemma 4.1, there exists 𝜔1>0 such that Δ𝑘≥𝜔1>0.(4.5)
On the other hand, |ğ’ž|<+∞, let 𝑁 be the last successful iteration, then 𝑍𝑁+1=𝑍𝑁+2=⋯=𝑍𝑁+𝑗(𝑗≥1) are unsuccessful iterations. From Steps 4 and 5 of Algorithm 3.8, 𝑟𝑁+𝑗<𝜂1, for sufficiently large 𝑁, we have limğ‘â†’âˆžÎ”ğ‘+𝑗=0,(4.6) which contradicts (4.5). The proof is completed.

We now consider what happens if the set 𝒜 is infinite in the course of Algorithm 3.8.

Theorem 4.4. Suppose that |ğ’ž|=|𝒜|=+∞, assumptions (S1) and (S2) hold. For any 𝜏>0 and 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛, if 𝑋 and 𝑆 are nonsingular, then each accumulation point of the infinite sequences generated by Algorithm 3.8 is a stationary point of Ψ𝜏(𝑍).

Proof. The proof is by contradiction. Suppose that {𝑍𝑘} is an infinite sequence generated by Algorithm 3.8, and any accumulation point of {𝑍𝑘} is not a stationary point of Ψ𝜏(𝑍). Suppose furthermore that 𝑍∗ and 𝜏∗ are the accumulation points of {𝑍𝑘} and {𝜏𝑘}, respectively. Since 𝑍∗ is not a stationary point of Ψ𝜏(𝑍), then ∇Ψ𝜏∗𝑍∗≠0(4.7) and there exists 𝜖0>0 such that ‖‖∇Ψ𝜏∗𝑍∗‖‖>𝜖0>0.(4.8) For some 𝜖∗>0, let 𝒩(𝑍∗,𝜖∗) be a neighborhood of 𝑍∗. From (4.8), there exists {𝑍𝑘}𝑘∈𝐾∈𝒩(𝑍∗,𝜖∗) such that ‖‖∇Ψ𝜏𝑘𝑍𝑘‖‖≥𝜖0>0,(4.9) where 𝐾⊆𝒜.
For 𝑚,𝑚+𝜈∈𝐾, because Ψ𝜏𝑘𝑍𝑘−Ψ𝜏𝑘+1𝑍𝑘+1≥𝜂1𝜑𝑘(0)−𝜑𝑘Δ𝑍𝑘,(4.10) we obtain that Ψ𝜏𝑚𝑍𝑚−Ψ𝜏𝑚+𝜈𝑍𝑚+𝜈=𝑚+𝜈𝑖=𝑚∈𝐾Ψ𝜏𝑖𝑍𝑖−Ψ𝜏𝑖+1𝑍𝑖+1≥𝜂1𝑚+𝜈𝑖=𝑚∈𝐾𝜑𝑘(0)−𝜑𝑘Δ𝑍𝑘≥𝜂1𝑚+𝜈𝑖=𝑚∈𝐾12â€–â€–âˆ‡Î¨ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–âŽ§âŽªâŽ¨âŽªâŽ©Î”min𝑘,â€–â€–âˆ‡Î¨ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–â€–â€–âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸ğ‘‡âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–âŽ«âŽªâŽ¬âŽªâŽ­â‰¥ğœ‚1𝑚+𝜈𝑖=𝑚∈𝐾12𝜖0⎧⎪⎨⎪⎩Δmin𝑘,𝜖0â€–â€–âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸ğ‘‡âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–âŽ«âŽªâŽ¬âŽªâŽ­.(4.11)
From (4.10), we know that Ψ𝜏𝑘(𝑍𝑘) is monotone decreasing and bounded below, which implies that Ψ𝜏𝑚(𝑍𝑚)−Ψ𝜏𝑚+𝑛(𝑍𝑚+𝜈)→0 for ğ‘šâ†’âˆž,𝑚∈𝐾. Thus, 𝜂1𝑚+𝜈𝑖=𝑚∈𝐾12𝜖0⎧⎪⎨⎪⎩Δmin𝑘,𝜖0â€–â€–âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸ğ‘‡âˆ‡ğ»ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–âŽ«âŽªâŽ¬âŽªâŽ­âŸ¶0.(4.12) As a result, we have limğ‘˜â†’âˆž,𝑘∈𝐾Δ𝑘=0.(4.13) By the update rule of Δ𝑘, there exists an infinite subsequence 𝐾⋆⊆𝐾, and we have that 𝑟𝑖≤𝜂1,limğ‘–â†’âˆžÎ”ğ‘–=0,𝑖∈𝐾⋆.(4.14) which contradicts 𝑘∈𝐾⊆𝒜. This completes the proof.

In what follows, we investigate the case where the number of iterations added to the filter 𝐹(𝑘) in the course of Algorithm 3.8 is infinite.

Theorem 4.5. Suppose that |ğ’ž|=|ℬ|=+∞ but |𝐴|<+∞, SDP problems (2.1) and (2.3) have strictly feasible points. Suppose furthermore that assumptions (S1) and (S2) hold. For any 𝜏>0 and 𝑍∈𝒳𝑛×ℛ𝑚×𝒳𝑛, if 𝑋 and 𝑆 are nonsingular, then limğ‘˜â†’âˆžâ€–â€–ğ»ğœî€·ğ‘ğ‘˜î€¸â€–â€–=limğ‘˜â†’âˆžâ€–â€–âˆ‡Î¨ğœğ‘˜î€·ğ‘ğ‘˜î€¸â€–â€–=0.(4.15)

Proof. First let {𝜏𝑘} be the sequence generated by Algorithm 3.8. From Lemma 4.2, we have limğ‘˜â†’âˆžğœğ‘˜=0,(4.16) which, together with assumption (S2), the desired result follows from [16, Lemma  3.1].

5. Numerical Experiments

In this section, we describe the results of some numerical experiments with the Algorithm 3.8 for the random sparse SDP considered in [13]. All programs are written in Matlab code and all computations are tested under Matlab 7.1 on Pentium 4.

In addition, in the computations, the following values are assigned to the parameters in the Algorithm: 𝜂1=0.1, 𝜂2=0.5, 𝜂3=0.8, 𝜇=0.1, 𝛾=0.2, 𝛾1=0.5, and 𝛾2=2. We also use the stopping criteria is being of 𝜀=10−8.

In the following Table 1, the first two columns give the size of the matrix 𝐶 and the dimension of the variable 𝑦. In the middle columns, “𝐹-time” denotes the computing time (in seconds), “𝐹-it.” denotes the numbers iteration, and “𝐹-obj.” defines the value of Ψ𝜏𝑘(𝑍𝑘) when our stopping criteria is satisfied. Some numerical results of [13] are shown in the last two columns.

𝑛 𝑚 𝐹 -time 𝐹 -it. 𝐹 -obj. 𝑅 -time 𝑅 -it.

300200003020 3 . 2 5 6 6 𝑒 − 1 2 6327
300250007223 5 . 1 6 9 7 𝑒 − 1 3 12729
400300006325 0 . 2 2 1 2 𝑒 − 1 2 11832
4004000015231 6 . 2 0 0 8 𝑒 − 1 4 20246
5003000017635 8 . 5 2 1 6 𝑒 − 1 6 20139
5004000010841 9 . 1 5 3 5 𝑒 − 1 5 19852
6002000032147 8 . 9 6 6 0 𝑒 − 1 7 48558
6006000029838 3 . 5 7 2 2 𝑒 − 1 6 34556

As shown in Table 1, all test problems have been solved just few iterations compared with [13]. Furthermore, this algorithm is less sensitive to the size of SDP problems. Comparatively speaking, our method is attractive and suitable for solving large-scale SDP problems.

6. Conclusions

In this paper, we have proposed a filter-trust-region method for SDP problems. Such a method offers a trade-off between the accuracy of solving the subproblems and the amount of work for solving them. Furthermore, numerical results show that our algorithm is attractive for large-scale SDP problems.


The authors would like to thank Professor Florian Jarre for his advice and guidance, Thomas David and Li Luo for their grateful help, and also the referees for their helpful comments. This work is supported by National Natural Science Foundation of China 10971162.


  1. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory, vol. 15 of SIAM Studies in Applied Mathematics, SIAM, Philadelphia, Pa, USA, 1994. View at: Publisher Site | Zentralblatt MATH
  2. M. X. Goemans, “Semidefinite programming in combinatorial optimization,” Mathematical Programming, vol. 79, no. 1–3, pp. 143–161, 1997. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  3. H. Wolkowicz, R. Saigal, and L. Vandenberghe, Handbook of Semidefinite Programming, Kluwer Acadamic Publishers, Dordrecht, The Netherlands, 2000.
  4. Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming, SIAM Studies in Applied Mathrmatics, SIAM, Philadelphia, Pa, USA, 1994.
  5. F. Alizadeh, J. Pierre, A. Haeberly, and M. L. Overton, “A new primal-dual interiorpoint method for semidefinite programming,” in Proceedings of the 5th SIAM Conference on Applied Linear Algebra, pp. 113–117, SIAM, Philadelphia, Pa, USA, 1994. View at: Google Scholar
  6. R. D. C. Monteiro, “Primal-dual path-following algorithms for semidefinite programming,” SIAM Journal on Optimization, vol. 7, no. 3, pp. 663–678, 1997. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  7. C. Helmberg, F. Rendl, R. J. Vanderbei, and H. Wolkowicz, “An interior-point method for semidefinite programming,” SIAM Journal on Optimization, vol. 6, no. 2, pp. 342–361, 1996. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  8. X.-Y. Zhao, D. Sun, and K.-C. Toh, “A Newton-CG augmented Lagrangian method for semidefinite programming,” SIAM Journal on Optimization, vol. 20, no. 4, pp. 1737–1765, 2010. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  9. K.-C. Toh, “Solving large scale semidefinite programs via an iterative solver on the augmented systems,” SIAM Journal on Optimization, vol. 14, no. 3, pp. 670–698, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  10. F. Jarre and F. Rendl, “An augmented primal-dual method for linear conic programs,” SIAM Journal on Optimization, vol. 19, no. 2, pp. 808–823, 2008. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  11. C. Kanzow and C. Nagel, “Semidefinite programs: new search directions, smoothing-type methods, and numerical results,” SIAM Journal on Optimization, vol. 13, no. 1, pp. 1–23, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  12. M. Kočvara and M. Stingl, “On the solution of large-scale SDP problems by the modified barrier method using iterative solvers,” Mathematical Programming, vol. 109, no. 2-3, pp. 413–444, 2007. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  13. J. Malick, J. Povh, F. Rendl, and A. Wiegele, “Regularization methods for semidefinite programming,” SIAM Journal on Optimization, vol. 20, no. 1, pp. 336–356, 2009. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  14. K.-C. Toh and M. Kojima, “Solving some large scale semidefinite programs via the conjugate residual method,” SIAM Journal on Optimization, vol. 12, no. 3, pp. 669–691, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  15. F. Leibfritz and E. M. E. Mostafa, “An interior point constrained trust region method for a special class of nonlinear semidefinite programming problems,” SIAM Journal on Optimization, vol. 12, no. 4, pp. 1048–1074, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  16. N. I. M. Gould, S. Leyffer, and P. L. Toint, “A multidimensional filter algorithm for nonlinear equations and nonlinear least-squares,” SIAM Journal on Optimization, vol. 15, no. 1, pp. 17–38, 2004. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  17. C. Helmberg, Semidefinite Programming For Combinatorial Optimization, Konard-Zuse-Zent rum für informationstechnik, Berlin, Germany, 2000.
  18. A. Fischer, “A special Newton-type optimization method,” Optimization, vol. 24, no. 3-4, pp. 269–284, 1992. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  19. A. Fischer, “A Newton-type method for positive-semidefinite linear complementarity problems,” Journal of Optimization Theory and Applications, vol. 86, no. 3, pp. 585–608, 1995. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  20. P. Tseng, “Merit functions for semi-definite complementarity problems,” Mathematical Programming, vol. 83, no. 2, pp. 159–185, 1998. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  21. X. Chen and P. Tseng, “Non-interior continuation methods for solving semidefinite complementarity problems,” Mathematical Programming, vol. 95, no. 3, pp. 431–474, 2003. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  22. D. Sun and J. Sun, “Semismooth matrix-valued functions,” Mathematics of Operations Research, vol. 27, no. 1, pp. 150–169, 2002. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  23. H. Jiang, M. Fukushima, L. Qi, and D. Sun, “A trust region method for solving generalized complementarity problems,” SIAM Journal on Optimization, vol. 8, no. 1, pp. 140–157, 1998. View at: Publisher Site | Google Scholar | Zentralblatt MATH
  24. W. Sun and Y. Yuan, Optimization Theory and Methods, Nonlinear Programming, Springer, New York, NY, USA, 2006.

Copyright © 2012 Aiqun Huang and Chengxian Xu. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.