An Approximate Proximal Bundle Method to Minimize a Class of Maximum Eigenvalue Functions

Abstract

We present an approximate nonsmooth algorithm to solve a minimization problem, in which the objective function is the sum of a maximum eigenvalue function of matrices and a convex function. The essential idea to solve the optimization problem in this paper is similar to the thought of proximal bundle method, but the difference is that we choose approximate subgradient and function value to construct approximate cutting-plane model to solve the above mentioned problem. An important advantage of the approximate cutting-plane model for objective function is that it is more stable than cutting-plane model. In addition, the approximate proximal bundle method algorithm can be given. Furthermore, the sequences generated by the algorithm converge to the optimal solution of the original problem.

1. Introduction

Bundle method is one of the efficient and promising methods for solving nonsmooth optimization problems. For example, a class of maximum eigenvalue function can be minimized by bundle method [1], bundle-filter method can be used to deal with nonsmooth convex constrained optimization problem [2], and penalized bundle method was proposed to solve nonsmooth optimization problem by Bonnans et al. [3]. Recently, a minimization problem for a class of constrained maximum eigenvalue function has been solved in [4] with the help of penalized bundle method. However, when constructing the cutting-plane model for the objective function, the formula 𝜕𝐹(𝑦)=𝛽^∗𝜕𝜆_max(𝐴(𝑦))+𝜕𝑔(𝑦) shows that the subdifferential of 𝜆_max(𝐴(𝑦)) is involved in [4]. Note that 𝜕𝜆_max(𝐴(𝑦)) is the face of 𝐶_𝑛 exposed by 𝐴(𝑦), where 𝐶_𝑛∶={𝑉∈𝑆_𝑛:𝑉⪰0,tr𝑉=1}. So 𝜕𝜆_max(𝐴(𝑦)) changes drastically when the multiplicity of the 𝜆_max(𝐴(𝑦)) is changed [5]. Thus 𝜕𝐹(𝑦) is unstable and it leads to the instability of cutting-plane model in [4]. In this paper, to avoid this drawback, we try to give a more stable approximate cutting-plane model for the objective function.

In view of a class of nonsmooth optimization problem of the form min_{𝑦∈𝑅𝑛}𝑓𝑦+𝑔𝑦, (P) where 𝑓(𝑦)∶=(𝜆_max∘𝐴)(𝑦) is the composite function of 𝜆_max and an affine mapping 𝐴, specifically 𝑅^𝑛∋𝑦⟼𝐴𝑦∶=𝐴₀+𝛽𝑦 (1) is affine, 𝐴₀∈𝑆^𝑛 and 𝛽 is a linear operator from 𝑅^𝑛 to 𝑆^𝑛; 𝑔(𝑦) is a nonsmooth convex function. We modify the elements in bundle and give an approximate proximal bundle method algorithm for (P). Our algorithm is established on the basic assumption that at least one approximate function value and one approximate subgradient at each point are available. And suppose ridom𝑓(𝑦)∩ridom𝑔(𝑦)≠∅.

The initial motivation for our present work lies in the following facts. The whole idea of so-called bundle methods can be concentrated on constructing a good approximation for the objective function. Reference [4] has solved this class of optimization problem with the help of cutting-plane model and penalized bundle method. However, the cutting-plane model in [4] is unstable. Our motivation is to construct a more stable approximate model for the objective function. We try to use the approximate subdifferential of the objective function 𝐹(𝑦)∶=𝑓(𝑦)+𝑔(𝑦)=(𝜆_max∘𝐴)(𝑦)+𝑔(𝑦). Note that in the equation 𝜕_𝜀𝐹(𝑦)=𝛽^∗𝜕_𝜀𝜆_max(𝐴(𝑦))+𝜕_𝜀𝑔(𝑦), obviously, 𝜕_𝜀𝜆_max(𝐴(𝑦)) is involved. And 𝜕_𝜀𝜆_max(𝐴(𝑦)) is no longer a face of 𝐶_𝑛; it is the intersection of 𝐶_𝑛 with the half-space {𝑉∈𝑆_𝑛:𝑉⋅𝐴(𝑦)≥𝜆_max(𝐴(𝑦))−𝜀}. In particular, almost all matrices in 𝜕_𝜀𝜆_max(𝐴(𝑦)) have rank 𝑛 (for 𝜀>0), whereas the rank of matrices in 𝜕𝜆_max(𝐴(𝑦)) is at most the multiplicity 𝑟, which is 1 for almost all 𝐴(𝑦) (so 𝜕_𝜀𝜆_max(𝐴(𝑦)) is relative stable); this also gives an idea of the big gap existing between these two convex sets. Therefore we decide to introduce an enlarged subdifferential [6] of 𝜆_max(𝐴(𝑦)) which can be regarded as an outer approximation of 𝜕𝜆_max(𝐴(𝑦)) and an inner approximation of 𝜕_𝜀𝜆_max(𝐴(𝑦)). Through the enlarged subdifferential of 𝜆_max(𝐴(𝑦)), we can easily get an enlarged subdifferential of the objective function 𝐹(𝑦). Using the approximate subgradient in the enlarged subdifferential to construct the approximate cutting-plane model for the objective function, there are two advantages: on one hand it can avoid the instability of the cutting-plane model which is constructed in [4], on the other hand it can avoid too much elements in 𝜕_𝜀𝜆_max(𝐴(𝑦)). After constructing a more stable approximate cutting-plane model, we propose our algorithm and prove the convergence of the algorithm.

The rest of this paper is organized as follows. Section 2 mainly contains the approximate cutting-plane model of the objective function. Here we firstly introduce an enlarged subdifferential which is an outer approximation of 𝜕𝐹(𝑦), simultaneously, it is an inner approximation of 𝜕_𝜀𝐹(𝑦). And then, using the approximate subgradient in the enlarged subdifferential to complete the constructing of an approximate cutting-plane model for the objective function. Section 3 gives the algorithm with respect to the approximate proximal bundle method and also provides a corresponding compression mechanism. Section 4 is devoted to convergence analysis of the algorithm mentioned in Section 3. Section 5 gives the conclusions.

In the paper, the standard norm and inner product are all in Hilbert space and are denoted by ‖⋅‖ and ⟨⋅,⋅⟩.

2. The Approximate Model of the Objective Function

In this section, we will mainly give the approximate model for the objective function. It is known that [4] has given a kind of approximate model for the objective function 𝐹(𝑦): 𝐹_𝑘𝑦=max_{𝑖=1,2,…,𝑘}𝐹𝑥^𝑖+𝛽^∗𝑄𝐴𝑥^𝑖𝑍𝑄𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑖, (2) where 𝑄(𝐴(𝑥^𝑖))𝑍𝑄(𝐴(𝑥^𝑖))^𝑇∈𝜕𝜆_max(𝐴(𝑥^𝑖)) and 𝑠^𝑖∈𝜕𝑔(𝑥^𝑖).

It is known that 𝜕𝜆_max(𝐴(𝑦)) is the face of 𝐶_𝑛 exposed by 𝐴(𝑦), where 𝐶_𝑛∶={𝑉∈𝑆_𝑛:𝑉⪰0,tr𝑉=1}. If the multiplicity of 𝜆_max(𝐴(𝑦)) is changed, the subdifferential of 𝜆_max(𝐴(𝑦)) changes drastically. So it leads that the subdifferential of function 𝐹(𝑦) is unstable. Thus this causes the instability of 𝐹_𝑘(𝑦). In order to construct a stable approximate model, we take the approximate subdifferential of function 𝐹(𝑦) into account. Firstly, we consider the approximate subdifferential of 𝜆_max(𝐴(𝑦)).

Definition 1. For all 𝑦∈𝑅^𝑛, 𝜀≥0 and 𝐴:𝑅^𝑛→𝑆^𝑛 is an affine mapping. One defines the following:(1)the set of indices of 𝜀-largest eigenvalues: 𝐼_𝜀𝐴𝑦∶=𝑖∈1,2,…,𝑛∶𝜆_𝑖𝐴𝑦>𝜆_max𝐴𝑦−𝜀; (3)(2)the 𝜀-multiplicity of 𝜆_max(𝐴(𝑦)): 𝑟_𝜀∶=max𝑖∶𝑖∈𝐼_𝜀𝐴𝑦; (4)(3)the 𝜀-first eigenspace: 𝐸_𝜀𝐴𝑦∶=_{𝑖∈𝐼𝜀(𝐴(𝑦))}𝐸_𝑖𝐴𝑦, (5)where 𝐸_𝑖(𝐴(𝑦)) is the eigenspace of 𝐴(𝑦) associated with the its eigenspace 𝜆_𝑖(𝐴(𝑦)).

For the composite maximum eigenvalue function, its approximate subdifferential is 𝜕_𝜀𝜆_max𝐴𝑦=𝑉∈𝐶_𝑛:𝑉⋅𝐴𝑦≥𝜆_max𝐴𝑦−𝜀, (6) where 𝐶_𝑛={𝑉∈𝑆^𝑛:𝑉⪰0,tr𝑉=1}. From this one sees that 𝜕_𝜀𝜆_max(𝐴(𝑦)) is the intersection of 𝐶_𝑛 with the half-space {𝑉∈𝑆^𝑛:𝑉⋅𝐴(𝑦)≥𝜆_max(𝐴(𝑦))−𝜀}, instead of being a face of 𝐶_𝑛. In particular, almost all matrices in 𝜕_𝜀𝜆_max(𝐴(𝑦)) have rank 𝑛 (for 𝜀>0), whereas the rank of matrices in 𝜕𝜆_max(𝐴(𝑦)) are at most the multiplicity 𝑟, which is 1 for almost all 𝐴(𝑦); this gives ideas that 𝜕_𝜀𝜆_max(𝐴(𝑦)) is more stable than 𝜕𝜆_max(𝐴(𝑦)) and there is a big gap existing between these two convex sets.

Introduce a compact convex set 𝜎_𝜀𝜆_max𝐴𝑦=𝑄_𝜀𝐴𝑦𝑍𝑄_𝜀𝐴𝑦^𝑇:𝑍∈𝐶_𝑟𝜀, (7) where 𝑄_𝜀(𝐴(𝑦)) is a 𝑛×𝑟_𝜀 matrice whose columns form orthonormal basis of 𝐸_𝜀(𝐴(𝑦)).

Proposition 2. Let 𝐴:𝑅^𝑛→𝑆^𝑛 be an affine mapping, then for all 𝑦∈𝑅^𝑛 and 𝜀≥0 one has 𝜕𝜆_max𝐴𝑦⊂𝜎_𝜀𝜆_max𝐴𝑦⊂𝜕_𝜀𝜆_max𝐴𝑦. (8)

Proof. The left inclusion can be derived from simple chain rules and (7). To see another inclusion, take 𝑃∈𝜎_𝜀𝜆_max(𝐴(𝑦)). Then 𝑃=𝑄_𝜀𝐴𝑦𝑍𝑄_𝜀𝐴𝑦^𝑇with𝑍∈𝐶_𝑟𝜀. (9) Since 𝑄_𝜀𝐴𝑦^𝑇𝑇𝑄_𝜀𝐴𝑦=diag𝜆_max𝐴𝑦,…,𝜆_𝑟𝜀𝐴𝑦⪰𝜆_𝑟𝜀𝐼_𝑟𝜀 (10) and tr𝑍=1. Thus it follows that 𝑃,𝐴𝑦=𝑍,𝑄_𝜀𝐴𝑦^𝑇𝑇𝑄_𝜀𝐴𝑦≥𝜆_𝑟𝜀. (11) Together with (3), we have ⟨𝑃,𝐴(𝑦)⟩≥𝜆_max(𝐴(𝑦))−𝜀; for 𝑃∈𝐶_𝑛 this means that, according to (6), 𝑃∈𝜕_𝜀𝜆_max(𝐴(𝑦)). Accordingly, the proof of right inclusion is completed.

Note that the compact convex set 𝜎_𝜀𝜆_max(𝐴(𝑦)) can be regarded as an outer approximation of 𝜕𝜆_max(𝐴(𝑦)) and an inner approximation of 𝜕_𝜀𝜆_max(𝐴(𝑦)). This set will avoid both the weakness of 𝜕𝜆_max(𝐴(𝑦)) (unstable) and the drawback of 𝜕_𝜀𝜆_max(𝐴(𝑦)) (too much elements). Applying the linear mapping 𝛽^∗ in (8). Then 𝛽^∗𝜕𝜆_max𝐴𝑦⊂𝛽^∗𝜎_𝜀𝜆_max𝐴𝑦⊂𝛽^∗𝜕_𝜀𝜆_max𝐴𝑦. (12) Set 𝜎_𝜀𝑓(𝑦)∶=𝛽^∗𝜎_𝜀𝜆_max(𝐴(𝑦)). Then we obtain 𝜕𝜆_max∘𝐴𝑦⊂𝜎_𝜀𝜆_max∘𝐴𝑦⊂𝜕_𝜀𝜆_max∘𝐴𝑦. (13) This formula is equivalent to 𝜕𝑓𝑦⊂𝜎_𝜀𝑓𝑦⊂𝜕_𝜀𝑓𝑦. (14) Add 𝜕𝑔(𝑦) simultaneously, since ridom𝑓(𝑦)∩ridom𝑔(𝑦)≠0, we have 𝜕𝐹𝑦=𝜕𝑓𝑦+𝜕𝑔𝑦⊂𝜎_𝜀𝑓𝑦+𝜕𝑔𝑦⊂𝜕_𝜀𝑓𝑦+𝜕_𝜀𝑔𝑦=𝜕_𝜀𝐹𝑦. (15) Set 𝜎_𝜀𝐹(𝑦)∶=𝜎_𝜀𝑓(𝑦)+𝜕𝑔(𝑦). Thus 𝜕𝐹(𝑦)⊂𝜎_𝜀𝐹(𝑦)⊂𝜕_𝜀𝐹(𝑦). Finally, 𝜎_𝜀𝐹(𝑦) is the enlarged subdifferential of the objective function 𝐹(𝑦) and it can be regarded as an outer approximation of 𝜕𝐹(𝑦) and an inner approximation of 𝜕_𝜀𝐹(𝑦). The following work will base on the enlarged subdifferential 𝜎_𝜀𝐹(𝑦).

Choose 𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇∈𝜎_𝜀𝑓(𝑥^𝑖) and 𝑠^𝑖∈𝜕𝑔(𝑥^𝑖), then 𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖∈𝜎_𝜀𝐹𝑥^𝑖⊂𝜕_𝜀𝐹𝑥^𝑖. (16) Next at each point 𝑥^𝑖 compute an approximate function value 𝐹_𝜀(𝑥^𝑖) satisfying 𝐹𝑥^𝑖−𝜀≤𝐹_𝜀𝑥^𝑖≤𝐹𝑥^𝑖. (17) Thus, using these approximate information of 𝐹(𝑦), the approximate model 𝐹_𝑘(𝑦) becomes 𝐹_𝑘𝑦=max_{𝑖=1,…,𝑘}𝐹_𝜀𝑥^𝑖+𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑖. (18) Let the aggregate linearzation error at 𝑥^𝑘 be denoted by 𝑒^𝑖: 𝑒^𝑖∶=𝐹𝑥^𝑘−𝐹_𝜀𝑥^𝑖−𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑥^𝑘−𝑥^𝑖. (19) With the notation, we obtain the form of approximate model 𝐹_𝑘𝑦=𝐹𝑥^𝑘+max_{𝑖=1,…,𝑘}−𝑒^𝑖+𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑘, (20) where 𝑍∈𝐶_𝑟𝜀.

Observe that 𝐹_𝑘(𝑦) is a more stable approximate model than 𝐹_𝑘(𝑦). In the following part, based on the approximate model 𝐹_𝑘(𝑦), we will give the approximate proximal bundle method algorithm.

3. The Approximate Proximal Bundle Method Algorithm

Algorithm 3 (the approximate proximal bundle method).
Step 0. Let 𝜃≥0, 𝑚∈(0,1) be given parameters. Choose 𝑥¹, we can obtain an approximate function value 𝐹_𝜀(𝑥¹) and approximate subgradient 𝛽^∗𝑄_𝜀(𝐴(𝑥¹))𝑍𝑄_𝜀(𝐴(𝑥¹))^𝑇+𝑠¹∈𝜎_𝜀𝐹(𝑥¹)⊂𝜕_𝜀𝐹(𝑥¹). Then construct the approximate model and set 𝑘=1, 𝛿₁=∞.
Step  1. If 𝛿_𝑘≤𝜃, stop.
Step  2. Solve the quadratic program: min_{𝑦∈𝑅𝑛}𝐹_𝑘𝑦+12𝜂_𝑘𝑦−𝑥^𝑘2. P₁ Define 𝛿_𝑘+1 as the nominal decrease: 𝛿_𝑘+1∶=𝐹𝑥^𝑘−𝐹_𝑘𝑦^𝑘+1−12𝜂_𝑘𝑦^𝑘+1−𝑥^𝑘2. (21)
Step  3. Call the black box again with 𝑦=𝑦^𝑘+1, if 𝐹𝑥^𝑘−𝐹𝑦^𝑘+1≥𝑚𝛿_𝑘+1. (22) Then set 𝑥^𝑘+1=𝑦^𝑘+1, otherwise set 𝑥^𝑘+1=𝑥^𝑘. Corresponding, the former one is said to be descent step and the latter one is called null step.
Step  4. Append 𝑦^𝑘+1 to the bundle model and construct 𝐹_𝑘+1. Change 𝑘 to 𝑘+1 and go to Step 1.

Remark 4. For a parameter 𝜂_𝑘>0, then the candidate point 𝑦^𝑘+1 can be obtained through solving the dual problem of P₁.

Theorem 5. If 𝑦^𝑘+1 is the unique solution to (𝑃₁) and suppose that 𝜂_𝑘>0, then 𝑦^𝑘+1=𝑥^𝑘−(1/𝜂_𝑘)∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖) and 𝛼=(𝛼₁,…,𝛼_𝑛𝑝𝑘) is a solution to 1min2𝜂_𝑘^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖+^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝑒^𝑖s.t.𝛼∈Δ_𝑘=𝛼_𝑖∈[0,1],^𝑛𝑝𝑘_𝑖=1𝛼_𝑖=1,𝑖=1,…,𝑛𝑝_𝑘. 𝐷₁ In addition, one can also have the following relations: (i)∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖)∈𝜕_𝜀𝐹_𝑘(𝑦^𝑘+1);(ii)𝛿_𝑘+1=𝜖_𝑘+(1/2𝜂_𝑘)‖∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖‖², where 𝜖_𝑘=∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝑒^𝑖;(iii)∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖)∈𝜕_𝜖𝑘𝐹(𝑥^𝑘).

Proof. Turn P₁ into a quadratic programming problem with an extra scalar variable 𝑟: min_{(𝑦,𝑟)∈𝑅𝑛×𝑅𝑟}1𝑟+2𝜂_𝑘𝑦−𝑥^𝑘2s.t.𝐹𝑥^𝑘−𝑒^𝑖+𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑘≤𝑟,𝑖=1,2,…,𝑛𝑝_𝑘. P₂ The corresponding lagrangian is, for 𝛼∈𝑅^𝑛𝑝𝑘₊, 𝐿1𝑦,𝛼,𝑟=𝑟+2𝜂_𝑘𝑦−𝑥^𝑘2+^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝐹𝑥^𝑘−𝑒^𝑖+𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑘−𝑟. (23) By strong convexity, the dual problem of P₁ is equivalent to problem 𝐷₁.
And 𝑦^𝑘+1=𝑥^𝑘−(1/𝜂_𝑘)∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖) is the solution to the quadratic programming problem P₁, then (i) holds.
To see (ii) and (iii), note that since there is no duality gap, the primal optimal value in P₁ equals the dual optimal value in 𝐷₁. Thus the term (ii) holds. The relation 𝐹(𝑦)≥𝐹_𝑘(𝑦)≥𝐹_𝑘(𝑦^𝑘+1) implies that the term (iii) holds.

From Theorem 5 we can ensure the candidate point 𝑦^𝑘+1 which appears in Step 2 can be obtained.

Remark 6. As iterations conduct, the elements in the bundle become more and more. When the size of the bundle becomes too big, it is necessary to compress bundle. So, at Step 4, one should append the compression subalgorithm.

When the current size of the bundle is bigger than the maximal size, that is, 𝑛𝑝_𝑘≥𝑛𝑝_max, Step 4 turns out to be as follows:

Step  4′. Let 𝑛_act∶={𝑖≤𝑛𝑝_𝑘:𝛼_𝑖≥0} be the active indices.

If 𝑛_act≤𝑛𝑝_max−1, then keep active couples and delete all inactive couples from the bundle. Set 𝑛_left=𝑛_act and define 𝑛𝑝_𝑘+1=𝑛_left+1. Then append new element 𝛽^∗𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1}𝑍𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1𝑇}+𝑠^{𝑛𝑝𝑘+1},𝑒^{𝑛𝑝𝑘+1} (24) to the bundle and construct model 𝐹_𝑘+1. Let 𝑘=𝑘+1, and go to Step 1. Note that 𝑒^{𝑛𝑝𝑘+1} is in new element: (i)when it is a descent step, 𝑒^{𝑛𝑝𝑘+1}=0,(ii)when it is a null step, 𝑒^{𝑛𝑝𝑘+1}=𝐹𝑥^𝑘−𝐹_𝜀𝑦^𝑘+1−𝛽^∗𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1}×𝑍𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1𝑇}+𝑠^{𝑛𝑝𝑘+1},𝑥^𝑘−𝑦^𝑘+1. (25)

If 𝑛_act>𝑛𝑝_max−1, then delete two or more couples of elements. In addition, 𝑛_left≤𝑛𝑝_max−2. Define 𝑛𝑝_𝑘+1=𝑛𝑝_left+2 and then append new element 𝛽^∗𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1}𝑍𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1𝑇}+𝑠^{𝑛𝑝𝑘+1},𝑒^{𝑛𝑝𝑘+1} (26) to the bundle and construct model 𝐹_𝑘+1. Let 𝑘=𝑘+1, and go to Step 1. Note that 𝑒^{𝑛𝑝𝑘+1} is in new element: (i)when it is a descent step, 𝑒^{𝑛𝑝𝑘+1}=0,(ii)when it is a null step, 𝑒^{𝑛𝑝𝑘+1}=𝐹𝑥^𝑘−𝐹_𝜀𝑦^𝑘+1−𝛽^∗𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1}𝑍𝑄_𝜀𝐴𝑥^{𝑛𝑝𝑘+1𝑇}+𝑠^{𝑛𝑝𝑘+1},𝑥^𝑘−𝑦^𝑘+1. (27)

Remark 7. Under certain circumstance with 𝑛𝑝_𝑘>𝑛𝑝_max, if the remaining couples are still too many after discarding all inactive couples from the bundle, one synthesizes indispensable information of active elements in bundle. Simultaneously the corresponding affine function is called aggregate linearization and is denoted by 𝐹_𝛼(𝑦)𝐹_𝛼𝑦∶=𝐹𝑥^𝑘−𝜖_𝑘+^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑘. (28) For the aggregate linearization 𝐹_𝛼(𝑦), it holds that 𝐹_𝛼𝑦=𝐹_𝑘𝑦^𝑘+1+^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑦^𝑘+1;𝐹_𝑘𝑦≥𝐹_𝛼𝑦−𝜀,∀𝑦∈𝑅^𝑛. (29) When the maximum capacity is reached, for instance, if 𝑘=𝑛𝑝_max, then assume that one discards the elements 𝑥¹,𝑥²,…,𝑥^𝑡(𝑡<𝑘) from the bundle and appends the aggregate couple. The resulting model will be 𝐹_𝑘+1𝑦=maxmax_{𝑡+1≤𝑖≤𝑘+1}𝐹_𝜀𝑥^𝑖+𝑚^𝑖,𝑦−𝑥^𝑖,𝐹_𝛼𝑦, (30) where 𝑚^𝑖=𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖. Note that for all 𝑘 and 𝑦∈𝑅^𝑛, in any case, one can have 𝐹_𝛼𝑦≤𝐹_𝑘+1𝑦≤𝐹𝑦;𝐹_𝑘+1𝑦≥𝐹𝑥^𝑘+1+𝛽^∗𝑄_𝜀𝐴𝑥^𝑘+1𝑍𝑄_𝜀𝐴𝑥^𝑘+1𝑇+𝑠^𝑘+1,𝑦−𝑦^𝑘+1. (31)

4. Convergence Analysis

To show convergence of the algorithm, we have to refer the stoping tolerance 𝜃. So we consider two situations, that is, 𝜃>0 and 𝜃=0.

Firstly, when the parameter 𝜃>0, use the notation 𝐾_𝑠 to denote the set of indices 𝑘 in which a new descent step is done.

Theorem 8. Consider Algorithm 3 and use the notation 𝐹^∗ to denote lim_{𝑘∈𝐾𝑠}𝐹(𝑥^𝑘). Assume that the algorithm never stops (𝑘→∞) as well as 𝐹^∗>−∞. Then 0≤_{𝑘∈𝐾𝑠}𝛿_𝑘≤𝐹𝑥¹−𝐹^∗𝑚. (32)

Proof. Since 𝜃>0 and the algorithm never stops, the nominal decrease must satisfy 𝛿_𝑘>0 for all 𝑘∈𝐾_𝑠. Note that 𝐾_𝑠 is a descent index set, we have 𝑥^𝑘+1=𝑦^𝑘+1 and 𝐹(𝑥^𝑘)−𝐹(𝑥^𝑘+1)=𝐹(𝑥^𝑘)−𝐹(𝑦^𝑘+1). Let 𝑘^′ be the index following 𝑘 in 𝐾_𝑠. Between 𝑘 and 𝑘^′ the algorithm makes null steps only without moving the stability center 𝑥^𝑘+1=𝑦^𝑘+𝑗 for all 𝑗=2,…,𝑘^′−𝑘. The descent test at 𝑘^′ gives 𝐹𝑥^𝑘+1−𝐹𝑥^𝑘′+1≥𝑚𝛿_𝑘′+1. (33) Thus for all 𝑘^′′∈𝐾_𝑠, it holds that when we let 𝑘^′′→∞, 𝑚^𝑘′′_{𝑘∈𝐾𝑠}≤^𝑘′′_{𝑘∈𝐾𝑠}𝐹𝑥^𝑘−𝐹𝑥^𝑘+1=𝐹𝑥¹−𝐹𝑥^𝑘′′+1=𝐹𝑥¹−𝐹^∗. (34) Removing 𝑚 to the right side of the inequation, we obtain the desired result.

When 𝜃 is taken strictly positive, by Theorem 8, there is an index 𝑘 for which 𝛿_𝑘≤𝜃 if (P) has minimizers. By Theorem 5(ii), both 𝜖_𝑘 and 1𝜂_𝑘last^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑘𝑍𝑄_𝜀𝐴𝑥^𝑘𝑇+𝑠^𝑘2 (35) are all small. Therefore, 𝑥^𝑘 is the minimizer.

Secondly, when the stopping tolerance 𝜃=0, the algorithm either stops by having found a solution to (P) or it never stops. In this case, there are two possibilities for the sequence of descent steps {𝑥^𝑘}_{𝑘∈𝐾𝑠}. One is that it has infinitely many elements. Another is that there is an iteration 𝑘 where a last descent step is done, that is, 𝑥^𝑘=𝑥^𝑘 for all 𝑘≥𝑘. We proof these two cases separately.

Case 1. There are infinitely many elements in 𝐾_𝑠.

Theorem 9. Suppose that the algorithm generates infinitely many descent steps 𝑥^𝑘 for all 𝑘∈𝐾_𝑠. One has (i)if (P) has an empty solution, then {𝑓(𝑥^𝑘)}↘0;(ii)if (P) has minimizers and ∑_{𝑘∈𝐾𝑠}(1/𝜂_𝑘)=+∞ with 𝑘∈𝐾_𝑠 and 𝑘→∞, the sequence {𝑥^𝑘} is minimizing for (P) and converges to a minimizer of (P).

Proof. (i) Note that the algorithm loops forever and 𝜃=0 for all 𝑘∈𝐾_𝑠, thus 𝛿_𝑘+1 holds. Then the infinite sequence of objective values {𝑓(𝑥^𝑘)} is strictly decreasing. If (P) has no solution, the sequence {𝑓(𝑥^𝑘)} is close to −∞. Therefore the proof of (i) is completed.
(ii) To show that {𝑥^𝑘} is a minimizing sequence. Suppose for contradiction purposes that there exists 𝑥∈𝑅^𝑛 and 𝑡>0 such that 𝐹(𝑥)≤𝐹(𝑥^𝑘)−𝑡 for all 𝑘∈𝐾_𝑠. By Theorem 5(iii), we have 𝑥−𝑥^𝑘,^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖≤𝐹(𝑥)−𝐹𝑥^𝑘+𝜖_𝑘. (36) By 𝑥^𝑘+1=𝑥^𝑘−(1/𝜂_𝑘)∑^𝑛𝑝𝑘_𝑖=1𝛼_𝑖(𝛽^∗𝑄_𝜀(𝐴(𝑥^𝑖))𝑍𝑄_𝜀(𝐴(𝑥^𝑖))^𝑇+𝑠^𝑖) and Theorem 5(ii), hence 𝑥^𝑘+1−𝑥²=𝑥^𝑘−1𝜂_𝑘^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖−𝑥²≤𝑥^𝑘−𝑥²+2𝜂_𝑘𝐹(𝑥)−𝐹𝑥^𝑘+𝜖_𝑘+2𝜂_𝑘𝛿_𝑘+1−𝜖_𝑘=𝑥^𝑘−𝑥²+2𝜂_𝑘𝐹(𝑥)−𝐹𝑥^𝑘+𝛿_𝑘+1. (37) Write the relation (37) for 𝑥=𝑥, we obtain 0≤𝑥^𝑘+1−𝑥²≤𝑥^𝑘−𝑥²+2𝜂_𝑘𝐹(𝑥)−𝐹𝑥^𝑘+𝛿_𝑘+1≤𝑥^𝑘−𝑥²+2𝜂_𝑘−𝑡+𝛿_𝑘+1. (38) Since {𝛿_𝑘}→0 with 𝑘→∞, there exists 𝑘_𝑡 such that 𝛿_𝑘+1<𝑡/2 for all 𝑘≥𝑘_𝑡. Thus the relation (38) becomes 0≤𝑥^𝑘+1−𝑥²≤𝑥^𝑘−𝑥²−𝑡𝜂_𝑘∀𝑘∈𝐾_𝑠,𝑘≥𝑘_𝑡. (39) Summing the inequations over 𝑘_𝑡≤𝑘∈𝐾_𝑠 yields 0≤𝑥^𝑘𝑡−𝑥²−𝑡_{𝑘≥𝑘𝑡,𝑘∈𝐾𝑠}1𝜂_𝑘. (40) Letting 𝑘→∞, we obtain a contradiction for the divergence assumption ∑(1/𝜂_𝑘)=+∞. Therefore, the sequence {𝑥^𝑘} is a minimizing sequence.
To see that {𝑥^𝑘} converges to a minimizer of (P), take in (37) 𝑥=𝑥, a solution to (P), and sum over 𝑘∈𝐾_𝑠. It holds that 𝐹(𝑥)≤𝐹(𝑥^𝑘) for all 𝑘. We see that 𝑥^𝑘−𝑥²≤𝑥¹−𝑥²+2𝜂_{min𝑘∈𝐾𝑠}𝛿_𝑘≤𝑥¹−𝑥²+2𝐹𝑥¹−𝐹^∗𝜂_min𝑚. (41) Then the sequence {𝑥^𝑘} is bounded. Take a subsequence {𝑥^𝑘𝑖}_𝑘^𝑖_∈𝐾𝑠 converging to 𝑥 as 𝑖→∞. Given 𝑢>0, take 𝑖 big enough such that 𝑥^𝑘𝑖−𝑥²≤𝑢2,_{𝑘∈𝐾𝑠,𝑘≥𝑘𝑖}𝛿_𝑘+1≤12𝑢𝜂_𝑘. (42) Writing (37) with 𝑥=𝑥 and summing it over 𝑘∈𝐾_𝑠 from 𝑘^𝑖 to an arbitrary 𝑘>𝑘^𝑖 yield 0≤𝑥^𝑘+1−𝑥²≤𝑥^𝑘𝑖−𝑥²+1𝜂_{𝑘𝑘∈𝐾𝑠,𝑘≥𝑘𝑖}𝛿_𝑘+1≤𝑢, (43) which implies that {𝑥^𝑘} converges to a minimizer of (P).

Case 2. There are finitely many descent steps. It means that the last descent step 𝑥^𝑘 is followed by an infinite number of null step, that is, 𝑥^𝑘=𝑥^𝑘 for all 𝑘≥𝑘.

Theorem 10. Assume that Algorithm 3 generates a last descent step 𝑥^𝑘 followed by infinitely many null steps. Then sequence {𝑥^𝑘} converges to 𝑥^𝑘 and 𝑥^𝑘 minimizes the function 𝐹.

Proof. Let 𝑦^𝑘+1 be the solution to P₁, we have 𝐹_𝑘𝑦^𝑘+1+12𝜂_𝑘𝑦^𝑘+1−𝑥^𝑘2≤𝐹_𝑘𝑥^𝑘≤𝐹𝑥^𝑘. (44) Set 𝑀_𝑘𝑦∶=𝐹_𝑘𝑦^𝑘+1+12𝜂_𝑘𝑦−𝑥^𝑘2+12𝜂_𝑘𝑦^𝑘+1−𝑦². (45) Then 𝑀_𝑘𝑦^𝑘+1≤𝐹𝑥^𝑘∀𝑘≤𝑘. (46) By (30), we have 𝑀_𝑘+1𝑦^𝑘+2≥𝑀_𝑘𝑦^𝑘+1+12𝜂_𝑘𝑦^𝑘+2−𝑦^𝑘+12;𝑀_𝑘𝑦^𝑘+1+12𝜂_𝑘𝑦^𝑘+1−𝑥^𝑘2=𝐹_𝑘𝑦^𝑘+1+𝜂_𝑘𝑦^𝑘+1−𝑥^𝑘2≤𝐹_𝛼𝑥^𝑘≤𝐹𝑥^𝑘. (47)(48) Thus, the sequence {𝑦^𝑘+1} is bounded. By the relation (47), we have 𝑦^𝑘+1−𝑦^𝑘⟶0as𝑘⟶∞. (49) It is obvious from the convexity that 𝐹_𝑘𝑦^𝑘+1−𝐹𝑦^𝑘≤𝐹𝑦^𝑘+1−𝐹𝑦^𝑘≤𝐿𝑦^𝑘+1−𝑦^𝑘2. (50) So, it holds that 𝐹_𝑘𝑦^𝑘+1−𝐹𝑦^𝑘⟶0. (51) From the bounded sequence {𝑦^𝑘} extract a subsequence {𝑦^𝑘𝑖}→𝑦 as 𝑖→∞, for {𝑦^𝑘+1−𝑦^𝑘}→0 and {𝑦^𝑘𝑖+1−𝑦}→0; thus 𝐹𝑦^𝑘𝑖+1−𝐹_𝑘^𝑖𝑦^𝑘𝑖+1⟶0as𝑖⟶∞. (52) Then 𝐹_𝑘^𝑖𝑦^𝑘𝑖+1⟶𝐹𝑦. (53) When 𝑘≥𝑘, the descent test is never satisfied, that is, 𝐹(𝑦^𝑘𝑖+1)−𝐹(𝑥^𝑘)>−𝑚𝛿_𝑘^𝑖₊₁, adding 𝛿_𝑘^𝑖₊₁ to the both sides of the inequation and using the definition of 𝛿_𝑘^𝑖₊₁ in (21), we can obtain 0≤(1−𝑚)𝛿_𝑘^𝑖₊₁≤𝐹𝑦^𝑘𝑖+1−𝐹_𝑘^𝑖𝑦^𝑘𝑖+1. (54) Let 𝑖→∞, then 𝛿_𝑘^𝑖₊₁→0. In addition, as 𝑖→∞𝜖_𝑘1⟶0,2𝜂_𝑘^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖2⟶0. (55) By Theorem 5(iii), it holds that 𝐹𝑦≥𝐹𝑥^𝑘+^𝑛𝑝𝑘_𝑖=1𝛼_𝑖𝛽^∗𝑄_𝜀𝐴𝑥^𝑖𝑍𝑄_𝜀𝐴𝑥^𝑖𝑇+𝑠^𝑖,𝑦−𝑥^𝑘−𝜖_𝑘^𝑖. (56) Passing to the limit as 𝑖→∞ the inequation shows that {𝑥^𝑘} minimizes the function 𝐹.
This is to show that 𝑦=𝑥^𝑘. Since 𝐹𝑦≥𝐹𝑥^𝑘≥𝐹_𝑘^𝑖𝑥^𝑘≥𝐹_𝑘^𝑖𝑦^𝑘𝑖+1+12𝜂_𝑘^𝑖₊₁𝑦^𝑘𝑖+1−𝑥^𝑘2≥𝐹_𝑘^𝑖𝑦^𝑘𝑖+1+12𝜂_𝑘𝑦^𝑘𝑖+1−𝑥^𝑘2, (57) passing to the limit as 𝑖→∞, we obtain 𝐹𝑦≥𝐹𝑦+12𝜂_𝑘𝑦−𝑥^𝑘2. (58) The inequality holds if and only if 𝑦=𝑥^𝑘. Thus 𝑥^𝑘 is the minimizer.

5. Conclusions

In this paper, we have given an enlarged subdifferential and constructed a more stable approximate cutting-plane model for the objective function. Then we have proposed an approximate proximal bundle algorithm and shown how to use this algorithm to minimize a class of maximum eigenvalue functions. Finally, it may be possible to extend this method to an even larger class of constrained maximum eigenvalue function.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

The authors would like to thank the referees for the valuable discussion and recommendation for improving the paper. This work is supported by the National Natural Science Foundation of China under Project no.11171138.