Particle Swarm Optimization-Proximal Point Algorithm for Nonlinear Complementarity Problems

Abstract

A new algorithm is presented for solving the nonlinear complementarity problem by combining the particle swarm and proximal point algorithm, which is called the particle swarm optimization-proximal point algorithm. The algorithm mainly transforms nonlinear complementarity problems into unconstrained optimization problems of smooth functions using the maximum entropy function and then optimizes the problem using the proximal point algorithm as the outer algorithm and particle swarm algorithm as the inner algorithm. The numerical results show that the algorithm has a fast convergence speed and good numerical stability, so it is an effective algorithm for solving nonlinear complementarity problems.

1. Introduction

The nonlinear complementarity problem is an important class of nonsmooth optimization problems and is widely used in mechanical, engineering, and economic fields. The algorithm research has attracted great attention from scholars at home and abroad. Algorithms for solving such problems mainly include the Lemke algorithm, homotopy method, projection algorithm, Newton algorithm, and interior point algorithm [1–10]. These algorithms were mostly based on the gradient method and relied on the selection of an initial point. How to select the appropriate initial point is a very difficult problem; therefore, in recent years, many scholars at home and abroad have been dedicated to the bionic intelligent algorithm not depending on the initial point and the gradient information. In 1995, Kennedy and Eberhart proposed the particle swarm algorithm (PSO) [11] of the predatory behavior of birds for the first time. It was similar to the genetic algorithm, an optimization tool based on iteration; its advantages were being easy to understand, easy to implement, and no need to adjust many parameters, and it was widely recognized by academic circles, who then put forward some improved algorithms. At present, the algorithm has been successfully applied in function optimization, neural network training, fuzzy system control, and so forth. In 2008, Zhang and Zhou [12] proposed a mixed algorithm for solving nonlinear minimax problems by combing the particle swarm optimization algorithm with the maximum entropy function method, and then Zhang extended the method to solve nonlinear L-1 norm minimization problems [13] and nonlinear complementarity problems [14] and achieved good results. But these algorithms were random algorithms depending on the probability essentially, where the entropy function was only playing the role of conversing problems. Sun et al. [15]. proposed a social cognitive optimization algorithm (SCO) based on entropy function for solving nonlinear complementarity problems. Yamashita et al. [16] proposed a new technique that utilizes a sequence generated with the PPA algorithm in 2004. And these algorithms cannot guarantee convergence to global optimal point with one hundred percent certainty even if the entropy function is a convex function. Proximal point algorithm is a global convergent algorithm for solving the convex optimization problem. Based on this, this paper researches a particle swarm optimization and proximal point algorithm for the discrete nonlinear complementary problem with every component function being a convex function, and the convergence to global optimal point with one hundred percent is guaranteed. First of all, the nonlinear complementary problem is transformed into an unconstrained optimization problem of a smooth function using the maximum entropy function, and then we construct the particle swarm optimization-proximal point algorithm by combing the particle swarm optimization algorithm with the proximal point algorithm, effectively giving full play to their respective advantages and optimizing the problem using the mixed algorithm. The numerical results show that the algorithm is a new effective algorithm with a fast convergence rate and good numerical stability.

2. The Smoothing of Nonlinear Complementary Problem

Let 𝑓∶𝑅^𝑛→𝑅^𝑛 be continuously differentiable; the nonlinear complementarity problem (NCP_(𝑓)) solves 𝑥∈𝑅^𝑛, in order to make 𝑥≥0,𝑓(𝑥)≥0,𝑥^𝑇𝑓(𝑥)=0, (1) when 𝑓(𝑥)=𝐌𝑥+𝑞 (where 𝐌 is an 𝑛×𝑛 matrix and 𝑞∈𝑅^𝑛 is constant vector), the NCP becomes linear complementarity problem, that is, LCP(𝑞,𝐌).

Definition 1 (see [2]). If 𝜑(𝑎,𝑏)∶𝑅²→𝑅 satisfies 𝜑(𝑎,𝑏)=0⟺𝑎≥0,𝑏≥0,𝑎𝑏=0, (2) then 𝜑(𝑎,𝑏) is an NCP function.

Common NCP functions are as follows:(1)𝜑(𝑎,𝑏)=𝑎²+𝑏²−𝑎−𝑏—Fisher-Burmeister function;(2)𝜑(𝑎,𝑏)=𝜆(𝑎²+𝑏²−𝑎−𝑏)+(1−𝜆)𝑎₊𝑏₊; 𝜆∈(0,1)—punishment Fisher-Burmeister function;(3)𝜑(𝑎,𝑏)=−𝑎𝑏+(1/2)min²{0,𝑎+𝑏}—differentiable NCP function.

We can transform solving the NCP(F) into solving the nonlinear equations by using Fisher-Burmeister function as follows: 𝜑𝑥₁,𝑓₁(𝑥)⋮𝜑𝑥_𝑛,𝑓_𝑛(𝑥)=0. (3) Equation (3) can be transformed into the optimization problem min_𝑥max_{1≤𝑖≤𝑛}𝜑𝑥_𝑖,𝑓_𝑖(𝑥), (4) where each component 𝜑(𝑥_𝑖,𝑓_𝑖(𝑥)) is smooth function of vector 𝑥∈Ω⊂𝑅^𝑛. However, the problem is a complex of nondifferentiable optimization problems.

We can transform the nonlinear complementary problem into smooth optimization problem by using the maximum entropy function method as follows.

Definition 2 (see [9, 10]). Let 𝐹_𝑝(𝑥)=1𝑝ln^𝑛_𝑖=1exp𝑝𝜑𝑥_𝑖,𝑓_𝑖(𝑥) (5) be maximum entropy function on 𝑥∈Ω⊂𝑅^𝑛 in (3); that is, 𝐹_𝑝(𝑥)=1𝑝ln^𝑛_𝑖=1exp𝑝𝜑𝑥_𝑖,𝑓_𝑖(𝑥)+exp−𝑝𝜑𝑥_𝑖,𝑓_𝑖(𝑥). (6)

Theorem 3 (see [9, 10]). Function 𝐹_𝑝(𝑥) goes along with the increase of parameter 𝑝 monotone decrease for any 𝑥∈Ω⊂𝑅^𝑛 and is limited to max_{1≤𝑖≤𝑛}{|𝜑(𝑥_𝑖,𝑓_𝑖(𝑥))|} (𝑝→∞); that is, 𝐹_𝑠(𝑥)≤𝐹_𝑟(𝑥),𝑠≤𝑟lim_𝑝→∞𝐹_𝑝(𝑥)=max_{1≤𝑖≤𝑛}𝜑𝑥_𝑖,𝑓_𝑖(𝑥). (7)

The initial ideas of entropy function method originated from the published paper of Kreisselmeier and Steinhauser [17] in 1979. The engineering and technical personnel like it at home and abroad because it is easy to prepare common software for solving many types of optimization problems, and it can provide the solution accuracy required with some convexity conditions. Since the 1980s, the method is widely used in structural optimization and engineering design and other fields. In recent years, the method gained the better results for solving constrained and unconstrained minimax problems, linear programming problems, and semi-infinite programming.

Large enough 𝑝 by the above theorem shows that we can use maximum entropy function instead of the objective function max_{1≤𝑖≤𝑛}{|𝜑(𝑥_𝑖,𝑓_𝑖(𝑥))|}; then the previous nonsmooth problems are changed into a differentiable function of unconstrained optimization problems. As is known to all, when 𝑝 is limited, we can get the approximate solution of the original problem, but when 𝑝 is appropriate we can guarantee high precision. For a problem with constraints, it can use the penalty function method into unconstrained problems.

3. The Idea of PSO Algorithm

PSO algorithm [11] is an evolutionary computation technique, originated from the behavior of birds preying. It is similar to the genetic algorithm, an optimization tool based on iteration. The system is initialized with a set of random solutions; through the iterative searching for the optimal value, there is no crossover and mutation in the genetic algorithm, but by searching with particles in solution space following the optimal particle, its optimization has been widely recognized by academe. In an 𝑛-dimensional target search space, there are 𝑚 particles forming a group, where the 𝑖th particle in the 𝑡th iteration is expressed as a vector 𝑥^𝑡_𝑖=(𝑥^𝑡_𝑖1,𝑥^𝑡_𝑖2,…,𝑥^𝑡_𝑖𝑛), 𝑖=1,2,…,𝑚. The position of each particle is a potential solution; flying speed is the 𝑛-dimensional vector 𝐕^𝑡_𝑖=(𝑣^𝑡_𝑖1,𝑣^𝑡_𝑖2,…,𝑣^𝑡_𝑖𝑛) correspondingly. In each iteration, note that 𝐏_𝑖𝑗=(𝑝_𝑖1,𝑝_𝑖2,…,𝑝_𝑖𝑛) is the optimal position of searching for the 𝑖th particle itself so far; 𝐏_𝑙𝑗=(𝑝_𝑙1,𝑝_𝑙2,…,𝑝_𝑙𝑛) is the optimal position of the whole particle swarm so far. In the (𝑡+1)th iteration, the speed and position update of the 𝑖th particle are 𝑣^𝑡+1_𝑖𝑛=𝑣^𝑡_𝑖𝑛+𝑐₁𝑟₁𝑝_𝑖𝑛−𝑥^𝑡_𝑖𝑛+𝑐₂𝑟₂𝑝_𝑙𝑛−𝑥^𝑡_𝑖𝑛𝑥_{𝑡+1𝑖𝑛},=𝑥^𝑡_𝑖𝑛+𝑣^𝑡+1_𝑖𝑛, (8)(9) where 𝑖=[1,𝑚], 𝑗=[1,𝑛], 𝑐₁ and 𝑐₂ are two learning factors, and 𝑟₁ and 𝑟₂ are pseudorandom numbers distributed on [0,1] Uniformly. 𝑣_𝑖𝑠∈[−𝑣_max,𝑣_max], where  𝑣_max is constant, set by the user themselves.

Shi and Eberhart presented the inertia weight PSO [18]: 𝑣^𝑡+1_𝑖𝑛=𝜔𝑣^𝑡_𝑖𝑛+𝑐₁𝑟₁𝑝_𝑖𝑛−𝑥^𝑡_𝑖𝑛+𝑐₂𝑟₂𝑝_𝑙𝑛−𝑥^𝑡_𝑖𝑛, (10) where 𝜔 is inertia weight in (10), controlling the previous velocity influence on the current speed. When 𝜔 is bigger, the influence of previous velocity is greater, and ability of global search of algorithm is weak; when 𝜔 is smaller, the influence of previous velocity is smaller, and ability of global search of algorithm is stronger. We can jump out of local minima by adjusting the size of 𝜔.

Equation (9) was improved in [19]: 𝑥^𝑡+1_𝑖𝑠=𝑥^𝑡_𝑖𝑠+(rand[]+𝑘)𝑣^𝑡+1_𝑖𝑠+10⁻⁶rand[]. (11)

Termination condition is the minimum adaptive threshold predetermined, which meets the maximum number of iterations or the optimal location searched by particle swarm according to the specific problems.

4. The Idea of PPA Algorithm

In 1970, Martinet [20] proposed the proximal point algorithm, which is a global convergence algorithm for a convex optimization problem; then Rockafellar [21] researched and extended the algorithm.

Consider the following optimization problem: min_𝑥,𝑦𝑓𝑥,𝑦, (12) where 𝑓∶𝑅^𝑛+𝑚→(−∞,+∞] is a closed regular convex function.

Proximal point algorithm (PPA) produces the iterative sequence {(𝑥_𝑘,𝑦_𝑘)} for solving (12), which is as follows: 𝑥₀,𝑦₀∈𝑅^𝑚+𝑛₊₊𝑥_𝑘+1,𝑦_𝑘+1=arg_{𝑥,𝑦∈𝑅𝑚+𝑛++}min𝑓𝑥,𝑦+𝜆_𝑘𝐷𝑥,𝑦,𝑥_𝑘,𝑦_𝑘, (13) where 𝐷(⋅,⋅) is a distance function; the proximal point algorithm uses 𝐷(⋅,⋅)=(1/2)‖(𝑥,𝑦)−(𝑥_𝑘,𝑦_𝑘)‖² early. In recent years, many scholars put forward similar distance functions to meet the convexity, such as Bregman function, similar to entropy function.

Proximal point algorithm is a classical deterministic algorithm. If the optimization problem is a convex optimization problem, the iterative sequence {(𝑥_𝑘,𝑦_𝑘)} produced by the algorithm converges to the global optimal point. This paper assumes that each component of the nonlinear complementarity problem (12) is a convex function; then the corresponding entropy function is a convex function; thus, we can ensure that the iterative sequences converge to global optimal with proximal point algorithm as the outer algorithm.

5. Particle Swarm Optimization-Proximal Point Algorithm for Nonlinear Complementarity Problems

Combined with the proximal point algorithm, the particle swarm optimization-proximal point algorithm for nonlinear complementarity problems is as follows.

5.1. The Steps of the Outer Algorithm (See Figure 1)

(1)Take initial point for 𝑥₀, the parameters 𝑝>0, 𝜆_𝑘>0; let 𝑘∶=1.(2)Take PSO algorithm in order to solve the problem of adjacent point iteration, we get 𝑥_𝑘+1.(3)Check whether it has reached the requirements of precision, if it has, the algorithm stops; otherwise, 𝑘∶=𝑘+1, to 2.

Figure 1 

5.2. The Inner Particle Swarm Algorithm

(1)Initialize the particle swarm of 𝑁, and set the initial position and velocity of each particle.(2)Calculate fitness value of each particle.(3)Compare the fitness value of each particle with the optimal position 𝑃_𝑖𝑗 experienced if good, as the current optimal position.(4)Compare the fitness value of each particle with the optimal position 𝑃_𝑖𝑗 global experienced if good, as the current global optimal position.(5)Evolve the position and speed of particle by (9) and (10).(6)If the termination condition is satisfied, we output the solution; otherwise, it returns to (2).

6. Numerical Results

We take a nonlinear complementary problem with four different functions in [2, 3, 5, 7] to verify the validity of the new algorithm, and the comparative results with those of [14–16] are as follows.

Example [2, 3, 5, 7]. Consider 𝑓₁(𝑥)=3𝑥²₁+2𝑥₁𝑥₂+2𝑥²₂+𝑥₃+3𝑥₄𝑓−6,₂(𝑥)=2𝑥²₁+𝑥₁+𝑥²₂+10𝑥₃+2𝑥₄𝑓−2,₃(𝑥)=3𝑥²₁+𝑥₁𝑥₂+2𝑥²₂+2𝑥₃+9𝑥₄𝑓−9,₄(𝑥)=𝑥²₁+3𝑥²₂+2𝑥₃+3𝑥₄−3. (14) The solution of the problem is (√6/2,0,0,1/2)^𝑇 and (1,0,3,0)^𝑇. Here, 𝑝 is 10⁵, the two learning factors are 𝑐₁=𝑐₂=2, where 𝜔 reduces from 1.0 to 0.4, group size is 20, the number of maximum evolution is 100, and the search range is [−2,2].

We programme the algorithm using VC++6.0 in Windows XP with CPU being Pentium 4 2.93 GHz and memory being 512 MB; the results are in Table 1, where the inner algorithm runs after 10 times, taking the worst solution as the initial iteration point in the next step, and required accuracy is 0.0001.

Table 1 

The results of PSO-PPA algorithm.

Data analysis: the algorithms in papers [2, 3, 5, 7] were all deterministic algorithms; therefore, there is no comparison with “the worst solution” and “success rate of searching” in Table 2. We calculate the maximum error by |𝐹_{𝑝(𝑥)−𝑓(𝑥)}|, where we take the worst solution in ten times as the calculation error. From Table 1, we know that it needed 5 iterations of outer circulation and 100 the maximum iteration times in inner circulation. The paper [14] needs 5000 generations of evolution, but the computing speed of our algorithm is much more faster than it.

Table 2 

The results of [14, 15] and PSO-PPA algorithm.

The pure PPA algorithm could not find the advantages probably for nonmonotonic NCP problems, which is proved in [16], so we find the near solution with PSO firstly and then solve iteratively by using PSO-PPA algorithm for nonmonotonic NCP problems. Compared with pure PPA algorithm, the PSO-PPA algorithm do not need the initial point, and its optimization speed is faster. The results are in Table 3.

Table 3 

The results of PSO-PPA algorithm and PPA algorithm [16]. Here, we take the initial point (4, 4, 4, 4), and 𝛼=0.8.

7. Conclusion

In this paper, first of all, we use the maximum entropy function method for smoothing and then propose a new efficient algorithm for solving nonlinear complementarity problems combined with the proximal point algorithm, without using the initial point and derivative information. This algorithm not only provides a new algorithm for nonlinear complementarity problems, but also expands the application range of the particle swarm algorithm. The experimental data shows that the algorithm has a fast convergence rate and good numerical stability and is an effective algorithm for nonlinear complementarity problems.

Acknowledgments

The work is being supported by the Scientific Research Foundation of the Education Department in Shaanxi Province of China (Grants nos. 11JK0493 and 12JK0887) and the Natural Science Basic Research Plan in Shaanxi Province of China (Grants nos. S2014JC9168 and S2014JC9190).