A QP-Free Algorithm for Finite Minimax Problems

Abstract

The nonlinear minimax problems without constraints are discussed. Due to the expensive computation for solving QP subproblems with inequality constraints of SQP algorithms, in this paper, a QP-free algorithm which is also called sequential systems of linear equations algorithm is presented. At each iteration, only two systems of linear equations with the same coefficient matrix need to be solved, and the dimension of each subproblem is not of full dimension. The proposed algorithm does not need any penalty parameters and barrier parameters, and it has small computation cost. In addition, the parameters in the proposed algorithm are few, and the stability of the algorithm is well. Convergence property is described and some numerical results are provided.

1. Introduction

We consider the nonlinear minimax problems of the form min_{𝑥∈𝑅𝑛}max_𝑗∈𝐼𝑓_𝑗(𝑥), (1) where 𝐼={1,2,…,𝑚} and 𝑓_𝑗(𝑗∈𝐼)∶𝑅^𝑛→𝑅 are continuously differentiable. Denote 𝐹(𝑥)=max𝑓_𝑗(𝑥),𝐼(𝑥)=,𝑗∈𝐼𝑗∈𝐼∣𝑓_𝑗(𝑥)(𝑥,𝑔=𝐹)_𝑗(𝑥)=∇𝑓_𝑗(𝑥),𝑗∈𝐼. (2) By introducing an auxiliary variable 𝑠∈𝑅, the problem (1) can be represented as the following standard nonlinear program (see [1]): min_{𝑥∈𝑅𝑛,𝑠∈𝑅}𝑠∣𝑓_𝑗(𝑥)−𝑠≤0,𝑗∈𝐼. (3) It is obvious that the KKT conditions of (3) are equivalent to _𝑗∈𝐼𝜆_𝑗𝑔_𝑗(𝑥)=0,_𝑗∈𝐼𝜆_𝑗𝜆=1;_𝑗𝑓_𝑗(𝑥)−𝐹(𝑥)=0,𝜆_𝑗≥0,𝑗∈𝐼. (4) So, a point 𝑥 is called the stationary point (see [2]) of (1), if there exists a vector 𝜆=(𝜆_𝑗,𝑗∈𝐼) such that (4) holds, where 𝜆 is called the multiplier vector.

1.1. Related Work

Some algorithms have been proposed to solve the minimax problems (1) and can be grouped into three classes. The first one is the direct nonsmooth method. The problem (1) is viewed as an unconstrained nonsmooth optimization problem, which can be solved by some nonsmooth methods, such as bundle methods, gradient sampling methods, and cutting plane methods; see, for example, [3–5].

Secondly, a variety of regularization approaches have been used to obtain smooth approximations to the problem (1); see, for example, [6–10]. The main advantage of the smoothing techniques is that the minimax problems are converted into smooth unconstrained optimization problems that can be solved by a standard unconstrained minimization solver. However, when the approximation accuracy is high, the smooth approximating problems become significantly ill-conditioned. Hence, the unconstrained optimization solver may experience numerical difficulty and slow convergence. Consequently, the simple use of smoothing techniques is complicated by the need of trading-off accuracy of approximation against problem ill-conditioning.

The third one is based on solving the equivalent smooth nonlinear programming problem (3), such as the sequential quadratic programming (SQP) methods (see [11–16]), the trust-region strategies (see [17, 18]), sequential quadratically constrained quadratic programming (SQCQP) method (see [19]), gradient projection method (see [20]), and the interior point (IP) methods (see [21–23]). The advantage of this class method is that the nondifferentiable optimization problem (1) can be transformed into an equivalent smooth constrained nonlinear programming problem which can be solved by well-established methods.

It is well known that the SQP method is one of the efficient methods for solving the smooth nonlinear program due to its fast convergence rate. Jian et al. [14], Hu et al. [15], and Zhu et al. [16] present some new SQP type algorithms for unconstrained minimax problems, respectively. For an iterative point 𝑥^𝑘, a new quadratic program (QP) subproblem is given by 1min𝑧+2𝑑^𝑇𝐻_𝑘𝑑s.t.𝑓_𝑗𝑥^𝑘+𝑔_𝑗𝑥^𝑘𝑇𝑑−𝐹𝑥^𝑘≤𝑧,𝑗∈𝐼_𝑘, (5) where 𝐻_𝑘 is a symmetric positive matrix and 𝐼_𝑘 is the 𝜀 active constraints set. The descent direction of 𝐹(𝑥) can be obtained by solving (5), and the algorithms have global and superlinear convergence by introducing another correction direction.

In these SQP algorithms or SQCQP method, QP subproblems or quadratically constrained quadratic programming (QCQP) subproblems with inequality constraints are also required to be solved which are computationally expensive compared with system of linear equations. In addition, the IP methods and the second class smoothing techniques do not need to solve QP and QCQP subproblems, but they need the penalty parameter or barrier parameter which is difficult to deal with and causes numerical difficulty when the penalty parameter or barrier parameter is too large. Therefore, it is necessary to construct a new algorithm without solving any QP and QCQP subproblems and using penalty parameters and barrier parameters.

1.2. Division of the Systems of Linear Equations

In this paper, we intend to replace the QP subproblem (5) used in [14–16] by two systems of linear equations with the same matrix so that the computation effort per iteration is much less and propose a QP-free algorithm, which does not need any penalty parameters and barrier parameters.

For (5), in order to speed up the rate of convergence and construct the systems of linear equations conveniently, we select an index 𝑗_𝑘∈𝐼(𝑥^𝑘) and consider the following QP subproblem by introducing more parameters 𝜂^𝑘_𝑗,𝑗∈𝐼_𝑘⧵{𝑗_𝑘}, associated with the iterate 𝑥^𝑘: 1min𝑧+2𝑑^𝑇𝐻_𝑘𝑑s.t.𝑔_𝑗𝑘𝑥^𝑘𝑇𝑓𝑑≤𝑧,_𝑗𝑥^𝑘+𝑔_𝑗𝑥^𝑘𝑇𝑑−𝐹𝑥^𝑘≤𝜂^𝑘_𝑗𝐼̃𝑧,𝑗∈_𝑘≜𝐼_𝑘⧵𝑗_𝑘. (6)

Since (6) is a convex program with linear constraints, its optimal solution is the KKT point; that is, (6) is equivalent to the following KKT system with the multiplier vector (𝜇,𝜆_𝑗,𝑗∈𝐼̃_𝑘): 𝜇+_{̃𝐼𝑗∈𝑘}𝜂^𝑘_𝑗𝜆_𝑗𝐻=1,_𝑘𝑑+𝜇𝑔_𝑗𝑘𝑥^𝑘+_{̃𝐼𝑗∈𝑘}𝜆_𝑗𝑔_𝑗𝑥^𝑘=0,0≤𝜇⟂−𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑+𝑧≥0,0≤𝜆_𝑗⟂−𝑓_𝑗𝑥^𝑘+𝐹𝑥^𝑘−𝑔_𝑗𝑥^𝑘𝑇𝑑+𝜂^𝑘_𝑗𝑧𝐼̃≥0,𝑗∈_𝑘. (7) Motivated by the KKT conditions above, we present the following system of linear equations (SLE): 𝜇+_{̃𝐼𝑗∈𝑘}𝜂^𝑘_𝑗𝜆_𝑗𝐻=1,_𝑘𝑑+𝜇𝑔_𝑗𝑘𝑥^𝑘+_{̃𝐼𝑗∈𝑘}𝜆_𝑗𝑔_𝑗𝑥^𝑘𝑔=0,_𝑗𝑘𝑥^𝑘𝑇𝑑−𝑧=0,𝑔_𝑗𝑥^𝑘𝑇𝑑−𝜂^𝑘_𝑗𝐼̃𝑧=0,𝑗∈_𝑘. (8) The equation “𝑔_𝑗𝑘(𝑥^𝑘)^𝑇𝑑−𝑧=0” comes from 𝜂^𝑘=(𝜂^𝑘_𝑗,𝑗∈𝐼̃_𝑘)→0(⇒𝜇→1). Obviously, the system above is equivalent to 𝐻_𝑘𝑑+_{̃𝐼𝑗∈𝑘}𝑔_𝑗𝑥^𝑘−𝜂^𝑘_𝑗𝑔_𝑗𝑘𝑥^𝑘𝜆_𝑗=−𝑔_𝑗𝑘𝑥^𝑘,𝑔_𝑗𝑥^𝑘−𝜂^𝑘_𝑗𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑=0,𝑗∈𝐼̃_𝑘. (9) Let (𝑑^𝑘0,𝜆_̃𝐼^𝑘0_𝑘) be the solution of (9), and 𝑑^𝑘0 is taken as the first direction of our algorithm, which is not entirely suitable as the main search direction although it is a descent direction of 𝐹(𝑥). In fact, the global convergence cannot be guaranteed, since the properties of the nonnegativity and complementarity of multiplier vector cannot be guaranteed at the point 𝑥^𝑘 with 𝑑^𝑘0 and at the accumulation points of the infinite sequence {𝑥^𝑘} generated by the proposed algorithm. Thus, we consider computing the second direction 𝑑^𝑘 by another linear system: 𝐻_𝑘𝑑+_{̃𝐼𝑗∈𝑘}𝑔_𝑗𝑥^𝑘−𝜂^𝑘_𝑗𝑔_𝑗𝑘𝑥^𝑘𝜆_𝑗=−𝑔_𝑗𝑘𝑥^𝑘,𝑔_𝑗𝑥^𝑘−𝜂^𝑘_𝑗𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑=𝑣^𝑘_𝑗,𝑗∈𝐼̃_𝑘, (10) where the right-hand parameters 𝑣^𝑘_𝑗,𝑗∈𝐼̃_𝑘, are yielded by 𝜆_̃𝐼^𝑘0_𝑘 and 𝐹(𝑥^𝑘)−𝑓_𝑗(𝑥^𝑘), 𝑗∈𝐼̃_𝑘 as follows: 𝑣^𝑘_𝑗=𝜆_𝑗^𝑘0,if𝜆_𝑗^𝑘0𝜆<0;_𝑗𝑘0𝐹𝑥^𝑘−𝑓_𝑗𝑥^𝑘,if𝜆_𝑗^𝑘0≥0. (11) Considering that the linear systems (10) and (9) have the same decomposed coefficient matrix, the computational cost is typically low. Lemma 8 shows that 𝑑^𝑘 is still a descent direction of 𝐹(𝑥), so 𝑑^𝑘 can be taken as the main search direction to design the algorithm.

The parameters 𝜂^𝑘_𝑗, 𝑗∈𝐼̃_𝑘, need to be devised deliberately to guarantee the nonsingularity of the coefficient matrix and global convergence. It is a difficult work throughout the whole research.

1.3. Properties of Our Algorithm

The proposed algorithm in this paper possesses the following properties.(i)Only the constraints indexed by some subset 𝐼_𝑘 of 𝐼 are considered which reduces the scale and computation cost of the subproblems to some extent.(ii)At each iteration, only the solutions of two linear systems with the same coefficient matrix are required; that is, the new algorithm is completely QP-free.(iii)It does not need any penalty parameters and barrier parameters. Therefore, the difficulty of choosing some suitable penalty parameters and barrier parameters is avoided.(iv)It needs few parameters which are adjusted easily, and the algorithm is robust.(v)It has weakly global convergence under some suitable assumptions.

We conclude this section by giving some notation which is used throughout this paper. The symbol ‖⋅‖ refers to the Euclidean norm. In addition, we denote by ∅ an empty set, the cardinality of any finite set 𝐽 by |𝐽|, and by det (𝑀) the determinant of the matrix 𝑀. Furthermore, the directional derivation of 𝐹 at the point 𝑥 along with the direction 𝑑 is denoted by 𝐹^′(𝑥;𝑑). It is easy to know that 𝐹^′(𝑥;𝑑)=max𝑔_𝑗(𝑥)^𝑇𝑑,𝑗∈𝐼(𝑥). (12)

2. Description of Algorithm

The new algorithm is based on the following assumption.(A1)The vectors {𝑔_𝑗(𝑥),𝑗∈𝐼(𝑥)⧵{𝑙}} are linearly independent for any 𝑙∈𝐼(𝑥) and each point 𝑥∈𝑅^𝑛.

Lemma 1. Suppose that the vectors {𝑔_𝑗(𝑥),𝑗∈𝐽} are linearly independent for 𝑥∈𝑅^𝑛 and index set 𝐽⊆𝐼. Then, for any given 𝑙∈𝐼, there exists a constant 𝜌_𝐽(𝑥)>0 such that {𝑔_𝑗(𝑥)−𝜁‖𝑔_𝑗(𝑥)‖𝑔_𝑙(𝑥),𝑗∈𝐽} are linearly independent for each 𝜁∈[0,𝜌_𝐽(𝑥)].

For a point 𝑥∈𝑅^𝑛, an index 𝑙∈𝐼, and a given index set 𝐽 such that {𝑔_𝑗(𝑥),𝑗∈𝐽} are linearly independent, we introduce the following technique similar to [24] to generate the parameter 𝜌_𝐽(𝑥) in Lemma 1. Define 𝑓_𝐽(𝑥)=𝑓_𝑗(𝑥),𝑗∈𝐽,𝑔_𝐽(𝑥)=𝑔_𝑗(𝑥),𝑛,𝑗∈𝐽_𝑗(𝑥)=𝑔_𝑗(𝑥)𝑔_𝑗(𝑥,𝑁)^𝐽(𝑥)=𝑛_𝑗(𝑥),𝑗∈𝐽,𝜌_𝐽(𝑥)=det𝑁_𝐽(𝑥)^𝑇𝑁_𝐽(𝑥)𝑒|𝐽|𝑔_𝑙(𝑥,)+1𝑎_𝑗(𝑥,𝜖)=𝑛_𝑗(𝑥)−𝜖𝜌_𝐽(𝑥)𝑔_𝑙(𝑥), (13)(14)(15) where 𝑒 is Napierian base and the parameter 𝜖≥0.

Lemma 2 (see [24]). Suppose that the vectors {𝑔_𝑗(𝑥),𝑗∈𝐽} are linearly independent. Then the vectors {𝑎_𝑗(𝑥,𝜖),𝑗∈𝐽} are also linearly independent for each 𝜖∈[0,1].

Let 𝑥^𝑘 be a given iteration point. We denote 𝑗_𝑘=min{𝑗∣𝑗∈𝐼(𝑥^𝑘)} and use the following pivoting operation to generate the index set 𝐼̃_𝑘 such that 𝑅_𝑘≜𝑔_̃𝐼𝑘(𝑥^𝑘) has full column rank, so vectors {𝑔_𝑗(𝑥^𝑘),𝑗∈𝐼̃_𝑘} are linearly independent.

Pivoting Operation (POP)

Step (i). Select an initial parameter 𝜀=𝜀_𝑘−1.

Step (ii). Generate 𝜀 active constraint subset 𝐼(𝑥^𝑘,𝜀) and matrix 𝐺_𝑘 by 𝐼𝑥^𝑘,𝜀=𝑗∈𝐼∣−𝜀≤𝑓_𝑗𝑥^𝑘−𝐹𝑥^𝑘,𝐺≤0_𝑘=𝑔_𝑗𝑥^𝑘,𝑗∈𝐼𝑥^𝑘,𝜀⧵𝑗_𝑘. (16)

Step (iii). If 𝐼(𝑥^𝑘,𝜀)⧵{𝑗_𝑘}=∅ or det(𝐺^𝑇_𝑘𝐺_𝑘)≥𝜀, then set 𝐼̃_𝑘∶=𝐼(𝑥^𝑘,𝜀)⧵{𝑗_𝑘}, 𝜀_𝑘=𝜀, and stop; otherwise, set 𝜀∶=𝜀/2 and repeat Step (ii).

For simplicity, denote 𝜌_𝑘≜𝜌_̃𝐼𝑘𝑥^𝑘,𝐴_𝐽𝑥^𝑘,𝜁=𝑔_𝑗𝑥^𝑘−𝜁𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘,𝑗∈𝐽,𝜁∈0,𝜌_𝑘. (17) From the POP and Lemma 2 above, we know that (𝑎_𝑗(𝑥^𝑘,𝜖),𝑗∈𝐼̃_𝑘) = (𝑛_𝑗(𝑥^𝑘)−𝜖𝜌_̃𝐼𝑘(𝑥^𝑘)𝑔_𝑗𝑘(𝑥^𝑘),𝑗∈𝐼̃_𝑘) has full column rank for each 𝜖∈[0,1]. Furthermore, it is easy to get the following lemma.

Lemma 3. Suppose that (A1) holds. Then the matrix 𝐴_𝐽(𝑥^𝑘,𝜁) has full column rank for each 𝐽⊆𝐼̃_𝑘 and 𝜁∈[0,𝜌_𝑘].

To describe some beneficial properties of the POP above, which is helpful for discussing the convergence of our algorithm, we have the following results.

Lemma 4. Suppose that (A1) holds, and let 𝑥^𝑘∈𝑅^𝑛.(i)The parameter 𝜀_𝑘 can be obtained in a finite number of steps in the POP.(ii)If a sequence {𝑥^𝑘} is bounded, then there exist two constants 𝜀,𝜀̃>0 such that inf𝜀_𝑘=𝜀,det𝑅^𝑇_𝑘𝑅_𝑘≥𝜀,∀𝑘,det𝐴_𝐽𝑘𝑥^𝑘,𝜁^𝑇𝐴_𝐽𝑘𝑥^𝑘,𝜁≥𝜀,̃∀𝑘,∀𝜁∈0,𝜌_𝑘,∀𝐽_𝑘⊆𝐼̃_𝑘. (18)(19)

Proof. Based on the assumption (A1), it is easy to get that (i) and (18) hold, so it is omitted here.
Now, we will prove that (19) holds. In view of 𝐼̃_𝑘 and 𝐽_𝑘 being the subsets of the fixed and finite set 𝐼 and the boundedness of {𝑥^𝑘}, we assume by contradiction without loss of generality that there exist an infinite index set 𝐾 and 𝜁_𝑘∈[0,𝜌_𝑘] such that 𝐼̃_𝑘≡𝐼̃,𝐽_𝑘≡𝐽⊆𝐼̃,𝑗_𝑘≡𝑗₀,𝑥^𝑘⟶𝑥^∗,det𝐴_𝐽𝑥^𝑘,𝜁_𝑘^𝑇𝐴_𝐽𝑥^𝑘,𝜁_𝑘⟶0,𝑘∈𝐾. (20) According to (14) and (A1), we can get 𝜌−−−→_𝑘^𝑘∈𝐾𝜌_∗=𝜌_̃𝐼(𝑥^∗). So, {𝜁_𝑘,𝑘∈𝐾} is bounded from 𝜁_𝑘∈[0,𝜌_𝑘]; we may assume that there exists an infinite index set 𝐾^′⊆𝐾 such that 𝜁−−−−→_𝑘^{𝑘∈𝐾′}𝜁_∗∈[0,𝜌_∗]. On the other hand, from (18) one knows that vectors {𝑔_𝑗(𝑥^∗),𝑗∈𝐼̃} are linearly independent. Therefore, by Lemma 2, we know that {𝑎_𝑗(𝑥^∗,𝜖),𝑗∈𝐼̃} are also linearly independent for each 𝜖∈[0,1], which implies that vectors 𝑔_𝑗𝑥^∗𝑎_𝑗𝑥^∗,𝜖=𝑔_𝑗𝑥^∗−𝜖𝜌_∗𝑔_𝑗𝑥^∗𝑔_𝑗0𝑥^∗,𝑗∈𝐼̃ (21) are linearly independent for each 𝜖∈[0,1]. So, {𝑔_𝑗(𝑥^∗)−𝜁‖𝑔_𝑗(𝑥^∗)‖𝑔_𝑗0(𝑥^∗),𝑗∈𝐽⊆𝐼̃} are linearly independent for each 𝜁∈[0,𝜌_∗]. Denote that 𝐴_∗=(𝑔_𝑗(𝑥^∗)−𝜁_∗‖𝑔_𝑗(𝑥^∗)‖𝑔_𝑗0(𝑥^∗),𝑗∈𝐽),𝐴_∗, has full column rank since 𝜁_∗∈[0,𝜌_∗]. However, 0=lim_{𝑘∈𝐾′}det𝐴_𝐽𝑥^𝑘,𝜁_𝑘^𝑇𝐴_𝐽𝑥^𝑘,𝜁_𝑘=det𝐴^𝑇_∗𝐴_∗, (22) which contradicts the fact that 𝐴_∗ has full column rank. So (19) holds.

A detailed description of the algorithm for solving (1) is given below.

Algorithm A. Parameters: 𝛼∈(0,1/2), 𝛽∈(0,1), 𝜀₋₁>0.
Step  0 (initialization). Consider 𝑥⁰∈𝑅^𝑛, an initial symmetric positive definite matrix 𝐻₀. Set 𝑘∶=0.
Step  1 (generating 𝜀 active set). Set parameter 𝜀=𝜀_𝑘−1, generate the set 𝐼̃_𝑘 by the POP, and let 𝜀_𝑘 be the corresponding termination parameter.
Step  2. Compute 𝜌_𝑘=𝜌_̃𝐼𝑘(𝑥^𝑘) according to (14), and adjust the parameter 𝜁_𝑘 by 𝜁_𝑘=𝜌₀𝜌,if𝑘=0;min_𝑘,𝑑^𝑘−1,0+𝑣^𝑘−1,𝜁_𝑘−1,if𝑘>0. (23)
Step  3. Compute the unique solution (𝑑^𝑘0,𝜆_̃𝐼^𝑘0_𝑘) of the following linear system: (LS1)𝑀𝑥^𝑘,𝐻_𝑘,𝜁_𝑘𝑑𝜆=−𝑔_𝑗𝑘𝑥^𝑘0, (24) with 𝑀𝑥^𝑘,𝐻_𝑘,𝜁_𝑘=𝐻_𝑘𝐴_𝑘𝐴^𝑇_𝑘0,𝐴_𝑘≜𝐴_̃𝐼𝑘𝑥^𝑘,𝜁_𝑘. (25)
Step  4. Compute the unique solution (𝑑^𝑘,𝜆^𝑘_̃𝐼𝑘) of the following linear system: (LS2)𝑀𝑥^𝑘,𝐻_𝑘,𝜁_𝑘𝑑𝜆=−𝑔_𝑗𝑘𝑥^𝑘𝑣^𝑘, (26) where 𝑣^𝑘=(𝑣^𝑘_𝑗,𝑗∈𝐼̃_𝑘) is yielded by 𝑣^𝑘_𝑗=𝜆_𝑗^𝑘0,if𝜆_𝑗^𝑘0𝜆<0;_𝑗𝑘0𝐹𝑥^𝑘−𝑓_𝑗𝑥^𝑘,if𝜆_𝑗^𝑘0≥0. (27) If 𝑑^𝑘=0 and 𝜆^𝑘_𝑗𝑘≜1−𝜁_{𝑘̃𝐼𝑗∈𝑘}𝜆^𝑘_𝑗𝑔_𝑗𝑥^𝑘≥0, (28) then 𝑥^𝑘 is a stationary point of (1), and stop.
Step  5 (doing line search). Compute the step size 𝑡_𝑘, the first number 𝑡 of the sequence {1,𝛽,𝛽²,…} satisfying 𝐹𝑥^𝑘+𝑡𝑑^𝑘≤𝐹𝑥^𝑘+𝛼𝑡𝐹^′𝑥^𝑘;𝑑^𝑘. (29)
Step  6. Set 𝑥^𝑘+1=𝑥^𝑘+𝑡_𝑘𝑑^𝑘, and compute a new symmetric positive definite matrix 𝐻_𝑘+1. Set 𝑘∶=𝑘+1, and go back to Step 1.

Remark 5. If the case 𝑑^𝑘=0, 𝜆^𝑘_𝑗𝑘<0 arises; the algorithm will stop at iterate 𝑥^𝑘 which is not a stationary point of (1). To avoid this pitfall, we will reset 𝜁_𝑘∶=(1/2)𝜁_𝑘 and solve (LS1) and (LS2) again.

For convenience of analysis in the rest of this paper, we give the equivalent forms of (LS1)-(LS2): ()⟺𝐻LS1_𝑘𝑑^𝑘0+_{̃𝐼𝑗∈𝑘}𝜆_𝑗^𝑘0𝑔_𝑗𝑥^𝑘−𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘=−𝑔_𝑗𝑘𝑥^𝑘,𝑔_𝑗𝑥^𝑘−𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘0𝐼̃=0,𝑗∈_𝑘;()⟺𝐻LS2_𝑘𝑑^𝑘+_{̃𝐼𝑗∈𝑘}𝜆^𝑘_𝑗𝑔_𝑗𝑥^𝑘−𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘=−𝑔_𝑗𝑘𝑥^𝑘,𝑔_𝑗𝑥^𝑘−𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘=𝑣^𝑘_𝑗,𝑗∈𝐼̃_𝑘. (30)(31) So, the parameters 𝜂^𝑘_𝑗,𝑗∈𝐼̃_𝑘, in (9)-(10) are selected as 𝜂^𝑘_𝑗=𝜁_𝑘‖𝑔_𝑗(𝑥^𝑘)‖. From Algorithm A and (A1) as well as Lemma 3, we can get the following lemmas immediately.

Lemma 6. The matrix 𝑀(𝑥^𝑘,𝐻_𝑘,𝜁) is nonsingular for each 𝜁∈[0,𝜌_𝑘]. Therefore, the coefficient matrix 𝑀(𝑥^𝑘,𝐻_𝑘,𝜁_𝑘) in systems (LS1)-(LS2) is nonsingular.

Taking into account the inverse matrix of 𝑀_𝑘≜𝑀(𝑥^𝑘,𝐻_𝑘,𝜁_𝑘) that can be expressed as 𝑀_𝑘⁻¹=^{𝐶𝑘𝐵𝑘}_{𝐵𝑇𝑘−𝐷𝑘−1}, with 𝐷_𝑘=𝐴^𝑇_𝑘𝐻_𝑘⁻¹𝐴_𝑘,𝐵_𝑘=𝐻_𝑘⁻¹𝐴_𝑘𝐷_𝑘⁻¹,𝐶_𝑘=𝐻_𝑘⁻¹−𝐻_𝑘⁻¹𝐴_𝑘𝐵^𝑇_𝑘, (32) from (LS1)-(LS2) and (32), we have the following relations: 𝑑^𝑘0=−𝐶_𝑘𝑔_𝑗𝑘𝑥^𝑘,𝜆_̃𝐼^𝑘0_𝑘=−𝐵^𝑇_𝑘𝑔_𝑗𝑘𝑥^𝑘;𝑑^𝑘=𝑑^𝑘0+𝐵_𝑘𝑣^𝑘,𝜆^𝑘_̃𝐼𝑘=𝜆_̃𝐼^𝑘0_𝑘−𝐷_𝑘⁻¹𝑣^𝑘. (33)(34)

Lemma 7. If Algorithm A stops at an iterate 𝑥^𝑘 with 𝑑^𝑘=0 and 𝜆^𝑘_𝑗𝑘≥0, then 𝑥^𝑘 is a stationary point of the problem (1).

Proof. If Algorithm A stops at an iterate 𝑥^𝑘 with 𝑑^𝑘=0 and 𝜆^𝑘_𝑗𝑘≥0, it follows from (31) that 1−𝜁_{𝑘̃𝐼𝑗∈𝑘}𝜆^𝑘_𝑗𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘+_{̃𝐼𝑗∈𝑘}𝜆^𝑘_𝑗𝑔_𝑗𝑥^𝑘𝑣=0,_𝑘=0. (35) By the definition of 𝑣^𝑘, we have 𝜆_̃𝐼^𝑘0_𝑘≥0, 𝜆_𝑗^𝑘0[𝑓_𝑗(𝑥^𝑘)−𝐹(𝑥^𝑘)]=0, 𝑗∈𝐼̃_𝑘, and hence, by (34), 𝜆^𝑘_̃𝐼𝑘=𝜆_̃𝐼^𝑘0_𝑘≥0,𝜆^𝑘_𝑗𝑓_𝑗𝑥^𝑘−𝐹𝑥^𝑘=0,𝑗∈𝐼̃_𝑘. (36) Since 𝑅_𝑘 has full column rank, from (35) it is easy to get 𝜆^𝑘_𝑗𝑘=1−𝜁_𝑘∑_{̃𝐼𝑗∈𝑘}𝜆^𝑘_𝑗‖𝑔_𝑗(𝑥^𝑘)‖≠0 and 𝜆^𝑘_𝑗𝑘>0. If one denotes 𝑢^𝑘_𝐼𝑘=(𝑢^𝑘_𝑗=𝜆^𝑘_𝑗/∑_{𝑗∈𝐼𝑘}𝜆^𝑘_𝑗,𝑗∈𝐼_𝑘), 𝑢^𝑘=(𝑢^𝑘_𝐼𝑘,0_{𝐼⧵𝐼𝑘}), then from (35)-(36) we can conclude that 𝑥^𝑘 is a stationary point of (1) with the multiplier vector 𝑢^𝑘.

Lemma 8. If 𝑑^𝑘≠0, then(i)𝑔_𝑗𝑘(𝑥^𝑘)^𝑇𝑑^𝑘=−(𝑑^𝑘0)^𝑇𝐻_𝑘𝑑^𝑘0−∑_{𝜆𝑗𝑘0<0}(𝜆_𝑗^𝑘0)²+∑_{𝜆𝑗𝑘0≥0}(𝜆_𝑗^𝑘0)²[𝑓_𝑗(𝑥^𝑘)−𝐹(𝑥^𝑘)]<0;(ii)𝑔_𝑗(𝑥^𝑘)^𝑇𝑑^𝑘≤𝜁_𝑘‖𝑔_𝑗(𝑥^𝑘)‖𝑔_𝑗𝑘(𝑥^𝑘)^𝑇𝑑^𝑘<0,𝑗∈𝐼(𝑥^𝑘)⧵{𝑗_𝑘};(iii)𝑑^𝑘 is a descent direction of 𝐹(𝑥) at the iterate 𝑥^𝑘, so the proposed algorithm is well defined.

Proof. (i) From (34), (30), (33), and (27), one gets 𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘=𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘0+𝐵_𝑘𝑣^𝑘=−𝑑^𝑘0𝑇𝐻_𝑘𝑑^𝑘0−𝜆_̃𝐼^𝑘0_𝑘^𝑇𝐴^𝑇_𝑘𝑑^𝑘0−𝜆_̃𝐼^𝑘0_𝑘^𝑇𝑣^𝑘=−𝑑^𝑘0𝑇𝐻_𝑘𝑑^𝑘0−𝜆_̃𝐼^𝑘0_𝑘^𝑇𝑣^𝑘=−𝑑^𝑘0𝑇𝐻_𝑘𝑑^𝑘0−_{𝜆𝑗𝑘0<0}𝜆_𝑗^𝑘02+_{𝜆𝑗𝑘0≥0}𝜆_𝑗^𝑘02𝑓_𝑗𝑥^𝑘−𝐹𝑥^𝑘. (37) Furthermore, since 𝑑^𝑘≠0, from (34), we can get (𝑑^𝑘0,𝑣^𝑘)≠(0,0). This implies that (i) holds.
(ii) If 𝑗∈𝐼(𝑥^𝑘)⧵{𝑗_𝑘}, from 𝐼(𝑥^𝑘)⧵{𝑗_𝑘}⊆𝐼̃_𝑘 and (27), we have 𝑣^𝑘_𝑗≤0. From Remark 5, we can assume without loss of generality that 𝑑^𝑘−1≠0. Furthermore, it follows from (34) and (23) that 𝜁_𝑘>0. Therefore, from the second formula of (31) and assumption (A1), it is easy to get 𝑔_𝑗𝑥^𝑘𝑇𝑑^𝑘=𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘+𝑣^𝑘_𝑗≤𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗𝑘𝑥^𝑘𝑇𝑑^𝑘𝑥<0,𝑗∈𝐼^𝑘⧵𝑗_𝑘. (38)
(iii) From (12) and the above results (i)-(ii), one knows that 𝐹^′(𝑥^𝑘;𝑑^𝑘)<0, which implies that 𝑑^𝑘 is a descent direction of 𝐹(𝑥) at 𝑥^𝑘. So the line search can be performed and the proposed algorithm is well defined.

3. Global Convergence

In this part, under mild assumptions, we show that Algorithm A is weak globally convergent; that is, there exists at least one accumulation point of the iterates {𝑥^𝑘} yielded by Algorithm A such that it is a stationary point of (1). To this end, in addition to (A1), the following two assumptions are necessary.(A2)The sequence {𝑥^𝑘} generated by Algorithm A is bounded.(A3)There exist two constants 𝑎,𝑏>0 such that 𝑎‖𝑑‖²≤𝑑^𝑇𝐻_𝑘𝑑≤𝑏‖𝑑‖²,∀𝑑∈𝑅^𝑛,∀𝑘. (39)

The following lemma establishes the boundedness of the associated sequences generated by the algorithm, and its proof is similar to Lemma  3.1 in [25], so it is omitted here.

Lemma 9. Suppose that (A1)–(A3) hold. Then, the sequences {𝜌_𝑘}, {𝜁_𝑘}, {𝑀_𝑘⁻¹}, {𝑑^𝑘0}, {𝜆_̃𝐼^𝑘0_𝑘}, {𝑑^𝑘}, {𝜆^𝑘_̃𝐼𝑘}, {𝐷_𝑘⁻¹}, and {𝐵_𝑘} are all bounded.

Lemma 10. Suppose that (A1)–(A3) hold. If an infinite index set 𝐾 satisfies 𝑥^𝑘⟶𝑥^∗,𝑗_𝑘≡𝑗^′,𝑔_𝑗′𝑥^𝑘𝑇𝑑^𝑘⟶0,𝑘∈𝐾, (40) then 𝑥^∗ is a stationary point of (1).

Proof. From Lemma 9, we know there exists an infinite subset 𝐾^′⊆𝐾 such that 𝐼̃_𝑘≡𝐼̃,𝑑^𝑘0⟶𝑑^∗0,𝜆_̃𝐼^𝑘0⟶𝜆_̃𝐼^∗0,𝑘∈𝐾^′. (41) Again, lim_𝑘→∞𝜁_𝑘=𝜁_∗ follows since {𝜁_𝑘} is monotone and bounded. In view of Lemma 8 (i) and (A3), we have 0⟵𝑔_𝑗′𝑥^𝑘𝑇𝑑^𝑘≤−𝑎𝑑^𝑘02−_{𝜆𝑗𝑘0<0}𝜆_𝑗^𝑘02+_{𝜆𝑗𝑘0≥0}𝜆_𝑗^𝑘02𝑓_𝑗𝑥^𝑘−𝐹𝑥^𝑘<0. (42) This along with (40)-(41) shows that 𝑑^𝑘0⟶0,𝜆_̃𝐼^∗0≥0,𝑘∈𝐾^′,𝜆_𝑗^∗02𝑓_𝑗𝑥^∗−𝐹𝑥^∗=0,𝑗∈𝐼̃. (43) So, passing to the limit for 𝑘∈𝐾^′, 𝑘→∞, in (30), one gets 1−𝜁_{∗̃𝐼𝑗∈}𝜆_𝑗^∗0𝑔_𝑗𝑥^∗𝑔_𝑗′𝑥^∗+_{̃𝐼𝑗∈}𝜆_𝑗^∗0𝑔_𝑗𝑥^∗𝜆=0;_𝑗∗0≥0,𝜆_𝑗^∗0𝑓_𝑗𝑥^∗−𝐹𝑥^∗=0,𝑖∈𝐼̃. (44) Furthermore, it follows from (23) and (27) that 𝜁_𝑘+1→0, 𝑘∈𝐾^′. So, 𝜁_∗=0 and 𝜆_𝑗^∗0_′≜1−𝜁_∗∑_{̃𝐼𝑗∈}𝜆_𝑗^∗0‖𝑔_𝑗(𝑥^∗)‖=1. If one denotes 𝐼^′=𝐼̃∪{𝑗^′}, 𝑢^∗_𝐼′=(𝑢^∗_𝑗=𝜆_𝑗^∗0/∑_{𝑗∈𝐼′}𝜆_𝑗^∗0,𝑗∈𝐼^′), and 𝑢^∗=(𝑢^∗_𝐼′,0_{𝐼⧵𝐼′}), then (44) implies that 𝑥^∗ is a stationary point with the multiplier vector 𝑢^∗.

Theorem 11. Suppose that (A1)–(A3) hold. Then Algorithm A either stops at a stationary point of (1) in a finite number of iterations or generates an infinite sequence {𝑥^𝑘}, of which at least one accumulation point is a stationary point of (1). In such sense, Algorithm A is said to possess weakly global convergence.

Proof. The cases 𝜁_∗=0 and 𝜁_∗>0 are discussed, respectively.
Case A. If 𝜁_∗=0, from (14), Lemma 4 (ii), and (A2), we know that there exists a constant 𝜌>0 such that 𝜌_𝑘≥𝜌 holds for all 𝑘 large enough. Therefore, in view of (A2) and (23), there exists an infinite index 𝐾 such that 𝑥^𝑘⟶𝑥^∗,𝜁_𝑘=𝑑^𝑘−1,0+𝑣^𝑘−1⟶0,𝑘∈𝐾; (45) that is, 𝑑^𝑘−1,0→0, 𝑣^𝑘−1→0, 𝑘∈𝐾. So, from (34) and the boundedness of {𝐵_𝑘}, we have lim_𝑘∈𝐾𝑑^𝑘−1=lim_𝑘∈𝐾(𝑑^𝑘−1,0+𝐵_𝑘−1𝑣^𝑘−1)=0. It further follows that ‖𝑥^𝑘−𝑥^𝑘−1‖=𝑡_𝑘−1‖𝑑^𝑘−1‖→0, 𝑘∈𝐾. Thus, lim_𝑘∈𝐾𝑥^𝑘−1=𝑥^∗. So, according to Lemma 10 and lim_𝑘∈𝐾𝑑^𝑘−1=0, it is easy to conclude that 𝑥^∗ is a stationary point of (1).
Case B. If 𝜁_∗>0, then we have 𝜁_𝑘≥(1/2)𝜁_∗ for all 𝑘∈𝐾 large enough. Suppose that 𝑥^∗ is any given accumulation point of {𝑥^𝑘}. Since 𝐼 is a fixed and finite set, from Lemma 9, there always exists an infinite index set 𝐾 such that 𝐼𝑥^𝑘≡𝐼_∗,𝐼̃_𝑘≡𝐼̃,𝑗_𝑘≡𝑗^′𝑥,𝑘∈𝐾,_𝑘,𝑑^𝑘0,𝑑^𝑘,𝜆_̃𝐼^𝑘0,𝜆^𝑘_̃𝐼,𝜁_𝑘⟶𝑥^∗,𝑑^∗0,𝑑^∗,𝜆_̃𝐼^∗0,𝜆^∗_̃𝐼,𝜁_∗,𝑘∈𝐾. (46) Suppose by contradiction that 𝑥^∗ is not a stationary point. We first show by contradiction that the given infinite index set 𝐾 satisfies 𝑔_𝑗′(𝑥^𝑘)^𝑇𝑑^𝑘→0, 𝑘∈𝐾. Otherwise, since 𝑔_𝑗′(𝑥^𝑘)^𝑇𝑑^𝑘<0 and 𝑔_𝑗′(𝑥^𝑘)^𝑇𝑑^𝑘→𝑔_𝑗′(𝑥^∗)^𝑇𝑑^∗, 𝑘∈𝐾, there exists a constant 𝑐>0 such that 𝑔_𝑗′𝑥^𝑘𝑇𝑑^𝑘≤−𝑐,𝑘∈𝐾largeenough. (47)
Consequently, we prove that the line search inequality (29) holds for all 𝑘∈𝐾 large enough and 𝑡>0 small enough. We denote 𝜔_𝑘(𝑡)=𝐹𝑥^𝑘+𝑡𝑑^𝑘−𝐹𝑥^𝑘−𝛼𝑡𝐹^′𝑥^𝑘,𝑑^𝑘. (48) It follows from the boundedness of {𝑑^𝑘} and Taylor expansion that 𝜔_𝑘(𝑡)=max𝑓_𝑗𝑥^𝑘+𝑡𝑑^𝑘−𝐹𝑥^𝑘−𝛼𝑡𝐹^′𝑥^𝑘,𝑑^𝑘𝑓,𝑗∈𝐼=max_𝑗𝑥^𝑘−𝐹𝑥^𝑘+𝑡𝑔_𝑗𝑥^𝑘𝑇𝑑^𝑘−𝛼𝑡𝐹^′𝑥^𝑘,𝑑^𝑘+𝑜(𝑡),𝑗∈𝐼. (49) Denote 𝑎_𝑘𝑗(𝑡)=𝑓_𝑗(𝑥^𝑘)−𝐹(𝑥^𝑘)+𝑡𝑔_𝑗(𝑥^𝑘)^𝑇𝑑^𝑘−𝛼𝑡𝐹^′(𝑥^𝑘,𝑑^𝑘)+𝑜(𝑡). For 𝑗∈𝐼_∗⧵{𝑗^′}⊆𝐼̃, according to (18) in Lemma 4 (ii), it is easy to know that there exists a constant 𝜎>0 such that ‖𝑔_𝑗(𝑥^𝑘)‖≥𝜎 for all 𝑘∈𝐾. From Lemma 8 (ii), one has for 𝑘∈𝐾 large enough and 𝑡>0 small enough 𝑎_𝑘𝑗(𝑡)=𝑡𝑔_𝑗𝑥^𝑘𝑇𝑑^𝑘−𝛼𝑡𝐹^′𝑥^𝑘,𝑑^𝑘(𝑡)≤()+𝑜1−𝛼𝑡𝑔_𝑗𝑥^𝑘𝑇𝑑^𝑘(𝑡)≤()+𝑜1−𝛼𝑡𝜁_𝑘𝑔_𝑗𝑥^𝑘𝑔_𝑗′𝑥^𝑘𝑇𝑑^𝑘(𝑡)1+𝑜≤−2(1−𝛼)𝜁_∗𝜎𝑐𝑡+𝑜(𝑡)≤0. (50) For 𝑗=𝑗^′, one has for 𝑘∈𝐾 large enough and 𝑡>0 small enough 𝑎_𝑘𝑗(𝑡)≤(1−𝛼)𝑡𝑔_𝑗′(𝑥^𝑘)^𝑇𝑑^𝑘+𝑜(𝑡)≤−(1−𝛼)𝑐𝑡+𝑜(𝑡)≤0.
For 𝑗∈𝐼⧵𝐼_∗, from the POP and Lemma 4 (ii), we can obtain 𝑓_𝑗(𝑥^𝑘)−𝐹(𝑥^𝑘)<−𝜀_𝑘≤−𝜀. So, 𝑎_𝑘𝑗(𝑡)≤−𝜀+𝑂(𝑡)≤0 holds for 𝑘∈𝐾 large enough and 𝑡>0 small enough.
Summarizing the analysis above, we know that there exists a constant 𝑡_∗>0 such that the step size 𝑡_𝑘≥𝑡_∗,∀𝑘∈𝐾. Furthermore, for ∀𝑗∈𝐼_∗, 𝑔_𝑗(𝑥^𝑘)^𝑇𝑑^𝑘≤−min{(1/2)𝜁_∗𝜎,1}𝑐. So, 𝐹^′(𝑥^𝑘,𝑑^𝑘)≤−min{(1/2)𝜁_∗𝜎,1}𝑐. Combining (29), we have 𝐹𝑥^𝑘+1−𝐹𝑥^𝑘≤𝛼𝑡_𝑘𝐹^′𝑥^𝑘,𝑑^𝑘≤−𝛼𝑡_∗1min2𝜁_∗𝜎,1𝑐,𝑘∈𝐾. (51) Since the whole sequence {𝐹(𝑥^𝑘)} is decreasing and lim_𝑘∈𝐾𝐹(𝑥^𝑘)=𝐹(𝑥^∗), we know that lim_𝑘→∞𝐹(𝑥^𝑘)=𝐹(𝑥^∗). So, passing to the limit for 𝑘∈𝐾 and 𝑘→∞ in inequality (51), one has −𝛼𝑡_∗min{(1/2)𝜁_∗𝜎,1}𝑐≥0, which contradicts the fact that 𝛼>0, 𝑡_∗>0, 𝜁_∗>0, and 𝑐>0. The contradiction shows that (40) holds. Furthermore, one can conclude from Lemma 10 that 𝑥^∗ is a stationary point of (1), which contradicts the assumption. The whole proof is completed.

4. Numerical Results

In this section, some preliminary numerical tests on 5 typical problems from [26] are reported, and the computation results show that Algorithm A is efficient. All the numerical experiments were implemented on MATLAB 7.0, under Windows XP and 2.2 GHz CPU. The BFGS formula with Powell’s modification [27] is adopted in the algorithm, and 𝐻₀ is the identity matrix. The parameters were selected as 𝛼=0.2, 𝛽=0.6, and 𝜀₋₁=1.2. In addition, execution is terminated if one of the following termination criteria is satisfied:(a)‖𝑑^𝑘‖≤10⁻⁵,(b)‖𝑥^𝑘+1−𝑥^𝑘‖≤10⁻⁵.

The computational results are reported in Table 1, and the columns of Table 1 have the following meanings: IP: the initial point; 𝑛: the number of variables; 𝑚: the number of functions 𝑓_𝑗(𝑥); ALG: the type of algorithm; NI: the number of iterations. “Algo A” represents Algorithm A in this paper, “J2006-1” and “J2006-2” represent the algorithms in [13], and “Hu2009” represents the algorithm in [15].

Table 1 

Numerical results of Algorithm A.

From Table 1, we can see that our algorithm can find the solutions of the test problems with a small number of iterations, and the computational results illustrate that our algorithm executes well for those problems. The numerical results are comparative with the algorithms in [13, 15]. Furthermore, we only need to solve two systems of linear equations with the same coefficient matrix per iteration. Considering that these linear systems have the same decomposed coefficient matrix, the computational cost per iteration of Algorithm A is typically low. This shows the potential advantage of our algorithm when applied to solving problems with large numbers of constraints. In addition, the parameters in the proposed algorithm are few, and the stability of the algorithm is very well.

5. Concluding Remarks

In this paper, a QP-free algorithm without solving any QP and QCQP subproblems is presented for unconstrained nonlinear finite minimax problems. At each iteration, only two systems of linear equations with the same coefficient matrix need to be solved. The proposed algorithm does not need any penalty parameters and barrier parameters which are difficult to deal with. Furthermore, under some mild assumptions, the global convergence is attained. As further work of this method, we think that there are still some problems worthy of discussing. For example, the assumption (A1) is different from the linearly independent assumption in common use, and any of them cannot derive the other. The discussion that the assumption (A1) is a constraint qualification needs further consideration. In addition, one should also take into account improving it to have superlinear convergence and generalizing it to solve minimax problems with inequality constraints.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (11271086), the Guangxi Natural Science Foundation (2013GXNSFBA019017, 2012GXNSFAA053007), and Science Foundation of Guangxi Education Department (2013YB080, ZD2014107).