Mathematical Problems in Engineering

Volume 2019, Article ID 4545064, 18 pages

https://doi.org/10.1155/2019/4545064

## Neural Network for Solving SOCQP and SOCCVI Based on Two Discrete-Type Classes of SOC Complementarity Functions

^{1}School of Science, Shenyang Aerospace University, Shenyang 110136, China^{2}Department of Mathematics, National Taiwan Normal University, Taipei 11677, Taiwan^{3}College of Mathematical Science, Inner Mongolia Normal University, Hohhot 010022, Inner Mongolia, China^{4}Department of Electrical Engineering, I-Shou University, Kaohsiung 840, Taiwan

Correspondence should be addressed to Jein-Shan Chen; wt.ude.untn@nehcsj

Received 15 November 2018; Revised 19 January 2019; Accepted 3 February 2019; Published 14 February 2019

Academic Editor: Sabri Arik

Copyright © 2019 Juhe Sun et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This paper focuses on solving the quadratic programming problems with second-order cone constraints (SOCQP) and the second-order cone constrained variational inequality (SOCCVI) by using the neural network. More specifically, a neural network model based on two discrete-type families of SOC complementarity functions associated with second-order cone is proposed to deal with the Karush-Kuhn-Tucker (KKT) conditions of SOCQP and SOCCVI. The two discrete-type SOC complementarity functions are newly explored. The neural network uses the two discrete-type families of SOC complementarity functions to achieve two unconstrained minimizations which are the merit functions of the Karuch-Kuhn-Tucker equations for SOCQP and SOCCVI. We show that the merit functions for SOCQP and SOCCVI are Lyapunov functions and this neural network is asymptotically stable. The main contribution of this paper lies on its simulation part because we observe a different numerical performance from the existing one. In other words, for our two target problems, more effective SOC complementarity functions, which work well along with the proposed neural network, are discovered.

#### 1. Introduction

In optimization community, it is well known that there are many computational approaches to solve the optimization problems such as linear programming, nonlinear programming, variational inequalities, and complementarity problems; see [1–6] and references therein. These approaches include the method using merit function, interior point method, Newton method, nonlinear equation method, projection method, and its variant versions. All the aforementioned methods rely on iterative schemes and usually only provide “approximate” solution(s) to the original optimization problems and do not offer real-time solutions. However, real-time solutions are eager in many applications, such as force analysis in robot grasping and control applications. Therefore, the traditional optimization methods may not be suitable for these applications due to stringent computational time requirements.

The neural network approach has an advantage in solving real-time optimization problems, which was proposed by Hopfield and Tank [7, 8] in the 1980s. Since then, neural networks have been applied to various optimization problems; see [9–30] and references therein. Unlike the traditional optimization algorithms, the essence of neural network approach for optimization is to establish a nonnegative Lyapunov function (or energy function) and a dynamic system that represents an artificial neural network. This dynamic system usually adopts the form of a first-order ordinary differential equation and its trajectory is likely convergent to an equilibrium point, which corresponds to the solution to the considered optimization problem.

Following the similar idea, researchers have also developed many continuous-time neural networks for second-order cone constrained optimization problems. For example, Ko, Chen and Yang [31] proposed two kinds of neural networks with different SOCCP functions for solving the second-order cone program; Sun, Chen, and Ko [32] gave two kinds of neural networks (the first one is based on the Fischer-Burmeister function and the second one relies on a projection function) to solve the second-order cone constrained variational inequality (SOCCVI) problem; Miao, Chen, and Ko [33] proposed a neural network model for efficiently solving general nonlinear convex programs with second-order cone constraints. In this paper, we are interested in employing neural network approach for solving two types of SOC constrained problems, the quadratic programming problems with second-order cone constraints (SOCQP for short) and the second-order cone constrained variational inequality (SOCCVI for short), whose mathematical formats are described as below.

The SOCQP is in the form ofwhere , is an matrix with full row rank, , and is the Cartesian product of second-order cones (SOCs), also called Lorentz cones. In other words, where are positive integers, , and denotes the SOC in defined bywith denoting the nonnegative real number set . A special case of (3) corresponds to the nonnegative orthant cone , i.e., and . We assume that is a symmetric positive semidefinite matrix and problem (1) satisfies a suitable qualification [34], such as the generalized Slater condition that there exists with strictly feasibility, then is a solution to problem (1) if and only if there exists a Lagrange multiplier such thatIn Section 3, we will employ two new families of SOC complementarity functions and use (4) to build up the neural network model for solving SOCQP.

We say a few words about why we assume that is a symmetric positive semidefinite matrix. First, it is clear that the symmetric assumption is reasonable because can be replaced by which is symmetric. Indeed, with being symmetric positive definite matrix, the SOCQP can be recast as a standard SOCP. To see this, we observe that which is done by completing the square. Then, the SOCQP (with ) is equivalent to which is also the same asThis formulation is further equivalent toNow, we let which says and denote Thus, the above reformulation (8) is expressed as a standard SOCP as follows:In view of this reformulation (10), we focus on SOCQP with being symmetric positive semidefinite in this paper.

The SOCCVI, our another target problem, is to find satisfyingwhere the set is finitely representable and is given byHere denotes the Euclidean inner product, , , and are continuously differentiable functions, and is a Cartesian product of second-order cones (or Lorentz cones), expressed aswith , , . When is affine, an important special case of the SOCCVI corresponds to the KKT conditions of the convex second-order cone program (CSOCP):where has full row rank, , , and . Furthermore, when is a convex twice continuously differentiable function, problem (14) is equivalent to the following SOCCVI which is to find such that where In fact, the SOCCVI can be solved by analyzing its KKT conditions:where is the variational inequality Lagrangian function, and . We also point out that the neural network approach for SOCCVI was already studied in [32]. Here we revisit the SOCCVI with different neural models. More specifically, in our earlier work [32], we had employed neural network approach to the SOCCVI problem (11) and (13), in which the neural networks were aimed at solving system (17) whose solutions are candidates of SOCCVI problem (11) and (13). There were two neural networks considered in [32]. The first one is based on the smoothed Fischer-Burmeister function, while the other one is based on the projection function. Both neural networks possess asymptotical stability under suitable conditions. In Section 4, in light of (17) again, we adopt new and different SOC complementarity functions to construct our new neural networks.

As mentioned earlier, this paper studies neural networks by using two new classes of SOC complementarity functions to efficiently solve SOCQP and SOCCVI. Although the idea and the stability analysis for both problems are routine, we emphasize that the main contribution of this paper lies on its simulations. More specifically, from numerical performance and comparison, we observe a new phenomenon different from the existing one in the literature. This may suggest update choices of SOC complementarity functions to work with neural network approach.

#### 2. Preliminaries

Consider the first-order differential equations (ODE):where is a mapping. A point is called an equilibrium point or a steady state of the dynamic system (18) if If there is a neighborhood of such that and , then is called an isolated equilibrium point.

Lemma 1. *Suppose that is a continuous mapping. Then, for any and , there exists a local solution to (18) with for some If, in addition, is locally Lipschitz continuous at , then the solution is unique; if is Lipschitz continuous in , then can be extended to .*

Let be a solution to dynamic system (18). An isolated equilibrium point is Lyapunov stable if for any and any , there exists a such that for all and . An isolated equilibrium point is said to be asymptotic stable if in addition to being Lyapunov stable, it has the property that as for all . An isolated equilibrium point is exponentially stable if there exists such that arbitrary point of (18) with the initial condition and is well defined on and satisfies where and are constants independent of the initial point.

Let be an open neighborhood of . A continuously differentiable function is said to be a Lyapunov function at the state over the set for (18) ifThe Lyapunov stability and asymptotical stability can be verified by using Lyapunov function, which is a useful tool for analysis.

Lemma 2. *(a) An isolated equilibrium point is Lyapunov stable if there exists a Lyapunov function over some neighborhood of .**(b) An isolated equilibrium point is asymptotically stable if there exists a Lyapunov function over some neighborhood of such that .*

For more details, please refer to any usual ODE textbooks, e.g., [35].

Next, we briefly recall some concepts associated with SOC, which are helpful for understanding the target problems and our analysis techniques. We start with introducing the Jordan product and SOC complementarity function. For any and , we define their Jordan product associated with as The Jordan product , unlike scalar or matrix multiplication, is not associative, which is a main source of complication in the analysis of SOC constrained optimization. There exists an identity element under this product, which is denoted by . Note that means and means the usual componentwise addition of vectors. It is known that for all . Moreover, if , then there exists a unique vector in , denoted by , such that . We also denote .

A vector-valued function is called an SOC complementarity function if it satisfiesThere have been many SOC complementarity functions studied in the literature; see [36–40] and references therein. Among them, two popular ones are the Fischer-Burmeister function and the natural residual function , which are given by Some existing SOC complementarity functions are indeed variants of and . Recently, Ma, Chen, Huang, and Ko [41] explored the idea of “discrete generalization” to the Fischer-Burmeister function which yields the following class of functions (denoted by ):where is a positive odd integer. Applying similar idea, they also extended to another family of SOC complementarity functions, , whose formula is as follows:where is a positive odd integer and means the projection onto . The functions and are continuously differentiable SOC complementarity functions with computable Jacobian, which can be found in [41].

#### 3. Neural Networks for SOCQP

In this section, we first show how we achieve the neural network model for SOCQP and prove various stabilities for it accordingly. Then, numerical experiments are reported to demonstrate the effectiveness of the proposed neural network.

##### 3.1. The model and Stability Analysis

As mentioned in Section 2, the KKT conditions are expressed in (4). With system (4) and using a given SOC complementarity function , it is clear to see that system (4) is equivalent to where . Moreover, we can specifically describe as the following: Here is a continuously differentiable SOC complementarity function such as and introduced in Section 2. It is clear that if solves , then solves . Accordingly, we consider a specific first-order ordinary differential equation as follows:where is a time scaling factor. In fact, letting , then . Hence, it follows from (28) that . In view of this, we set in the subsequent analysis. Next, we show that the equilibrium of the neural network (28) corresponds to the solution to system (4).

Lemma 3. *Let be an equilibrium of the neural network (28) and suppose that is nonsingular. Then solves system (4).*

*Proof. *Since and is nonsingular, it is clear to see that if and only if .

Besides, the following results address the existence and uniqueness of the solution trajectory of the neural network (28).

Theorem 4. *(a) For any initial point , there exists a unique continuously maximal solution with for the neural network (28).**(b) If the level set is bounded, then can be extended to .*

*Proof. *This proof is exactly the same as the one in [32, Proposition 3.4], so we omit it here.

Now, we are ready to analyze the stability of an isolated equilibrium of the neural network (28), which means and for , with being a neighborhood of .

Theorem 5. *Let be an isolated equilibrium point of the neural network (28). *(a)*If is nonsingular, then the isolated equilibrium point is asymptotically stable and hence Lypunov stable.*(b)*If is nonsingular for all , then the isolated equilibrium point is exponentially stable.*

*Proof. *The desired results can be proved by using Lemma 3 and mimicking the arguments as in [32, Theorem 3.1].

*3.2. Numerical Experiments*

*In order to demonstrate the effectiveness of the proposed neural network, we test three examples for our neural network (28). The numerical implementation is coded by Matlab 7.0 and the ordinary differential equation solver adopted here is ode23, which uses Ruge-Kutta formula. As mentioned earlier, in general the parameter is set to be . For some special examples, the parameter is set to be another value.*

*Example 6. *Consider the following SOCQP problem:

*After suitable transformation, it can be recast as an SOCQP with , , , and . This problem has an optimal solution . Now, we use the proposed neural network (28) with two cases and , respectively, to solve the above SOCQP and their trajectories are depicted in Figures 1–4. For the sake of coding needs and check, the following expressions are presented.*