Abstract
We propose a one-layer neural network for solving a class of constrained optimization problems, which is brought forward from the MDF continuous hot-pressing process. The objective function of the optimization problem is the sum of a nonsmooth convex function and a smooth nonconvex pseudoconvex function, and the feasible set consists of two parts, one is a closed convex subset of , and the other is defined by a class of smooth convex functions. By the theories of smoothing techniques, projection, penalty function, and regularization term, the proposed network is modeled by a differential equation, which can be implemented easily. Without any other condition, we prove the global existence of the solutions of the proposed neural network with any initial point in the closed convex subset. We show that any accumulation point of the solutions of the proposed neural network is not only a feasible point, but also an optimal solution of the considered optimization problem though the objective function is not convex. Numerical experiments on the MDF hot-pressing process including the model building and parameter optimization are tested based on the real data set, which indicate the good performance of the proposed neural network in applications.
1. Introduction
Medium density fibreboard (MDF) finds many applications in wood industries because of its favorable properties such as surface characteristics, dimensional stability, and excellent machinability [1, 2]. In the MDF hot-pressing process many physical processes are involved and the complexity of this operation arises from the fact that they are coupled. Hot-pressing process is one of the key procedures in the production of MDF, which influences the utilization ratio of energy and resource. With the decreased resource of timber and the increased demand of MDF, it is of great important to analyze the experimental data effectively and reasonably, find the main factors among the many indexes of MDF, and establish the relation models on the properties of slab, the parameters in the hot-pressing process, and the main indexes of MDF. These relation models not only can help the staff give reasonable prediction and a reliability assessment to the hot-pressing process according to the actual process parameters, but also provide a theoretical basis for the setting and adjusting of the main factors in hot-pressing process according to the actual demand of MDF properties. So optimization models and methods have been important tools for the optimization, control, and scheduling of the hot-pressing of plates.
Real-time online solutions of optimization problems are desired in many engineering and scientific applications. One possible and very promising approach to solve the real-time optimization problems is to apply artificial neural networks [3–5]. With the resemblance brains, neural networks can be implemented online by hardware and have become an important technical toll for solving optimization problems, for example, [3, 4, 6–9]. Based on the gradient method, the Hopfield neural networks proposed in [4, 5] are the two classical recurrent neural networks for linear and nonlinear programming, whereafter, in addition to the gradient method, many types of neural networks are designed, such as the Lagrangian neural networks [10], the projection-type neural networks [11, 12], the dual network [13], and the stochastic neural network [14]. Projection method is an effective and simple method for solving the constraints. However, it is impossible to solve the general constraints by projection method. Then, Lagrangian and penalty methods are introduced into networks. Based on the Lagrangian function method, Lagrangian networks were proposed for solving the optimization problems [8, 10] with general constraints. But the Lagrangian network increases the dimension of the networks along with the number of the constraints. In recent years, recurrent neural networks based on penalty method were widely investigated for solving optimization problems. The neural networks for smooth optimization problems can not solve nonsmooth optimization problems, because the gradients of the objective and constrained functions are required in such neural networks. The generalized nonlinear programming circuit (G-NPC) in [15] can be considered as a natural extension of nonlinear programming circuit (NPC) for solving nonsmooth convex optimization problems with inequality constraints. But the nonempty interior of feasible region and large enough penalty parameters are needed for the network in [15]. In order to overcome the nonempty assumption of the interior of feasible region, Bian and Xue [6] proposed a recurrent neural network for nonsmooth convex optimization based on penalty function method. The efficiency of the neural networks for solving convex optimization problems relies on the convexity of functions. A neural network for nonconvex quadratic optimization is presented in [16]. Some neural networks modeled by differential inclusion were also proposed for some nonsmooth and nonconvex optimization problems [6, 17]. To overcome the differential inclusion, smoothing techniques are introduced into the neural networks. The main feature of smoothing method is to approximate the nonsmooth functions by a class of smooth functions. Thus, the neural network constructed by the smoothing techniques is modeled by a differential equation, which can be implemented easily in circuits and mathematical software [18].
In this paper, we propose a neural network model for solving the optimization problem brought forward from the MDF continuous hot-pressing automatic control system. In Section 2, some notations and necessary preliminary results are listed. In Section 3, based on the SVM theory with the existing linear and nonlinear kernel functions, we give an optimization problem, which includes the problems for building the models of MDF continuous hot-pressing system and optimizing the MDF performance indexes as special cases. In order to build up the relation models on the properties of the slab, technical parameters in hot-pressing process, and the performance indexes of MDF, when the kernel function is positive definite or semipositive definite, the corresponding optimization problem is a constrained convex problem; otherwise it is a nonconvex problem. The optimization problem for optimizing the performance parameters is a nonconvex constrained optimization problem, but its objective function is pseudoconvex due to the appropriate choice of kernel functions. In Section 4, we propose a neural network based on the penalty function method, projection method, and smoothing techniques. The proposed network is modeled by a nonautomatic differential equation. By Lyapunov method, we prove that the solution of the proposed network is global existent and convergent to the feasible set of the considered optimization problems. Moreover, due to the pseudoconvexity of the objective function and the convexity of the constraint, the proposed network also converges to the optimal solution set of the optimization problem. In Section 5, based on the existing data set, we use the proposed network into the model building and parameter optimizing problems of hot-pressing system, which validates the good performance of the obtained results in this paper.
Notations. . Given column vectors and , is the scalar product of and . denotes the th element of . denotes the Euclidean 2-norm defined by . For a closed convex subset , is the distance from to defined by .
2. Preliminaries
In this section, we state some definitions and properties needed in this paper. We refer the readers to [19–21].
2.1. Support Vector Regression
Kernels were regarded as a function with the formulation of inner product and have been a powerful tool in machine learning for their superior performance over a wide range of learning problems, such as isolated hand written digit recognition, text categorization, and face detection [21, 22].
Let be a nonempty set and be a real-valued and symmetric function. With the kernel matrix , is said to be a positive semidefinite kernel, if is positive semidefinite for any and . We call an indefinite kernel, if there exist and such that and .
In what follows, we list some widely used kernels.(i)Gaussian radial basis function kernel: .(ii)Polynomial kernel: , , .(iii)Sigmoid kernel: , , .
2.2. Smoothing Approximation
Smoothing approximation is an effective method for solving nonsmooth optimization problems and has been widely used in the past decades. The main feature of smoothing method is to approximate the nonsmooth functions by a class of parameterized smooth functions. In this paper, we adopt the smoothing function defined as follows.
Definition 1 (see [23]). Let be a continuous function. One calls a smoothing function of , if is continuously differentiable for any fixed and holds for any .
Chen and Mangasarian constructed a class of smooth approximations of the function by convolution [20, 24] as follows. Let be a piecewise continuous density function satisfying Thenfrom to is well defined.
By different density functions, many popular smoothing functions of can be derived, such aswhere is the neural networks smoothing function, is called the CHKS (Chen-Harker-Kanzow-Smale) smoothing function, is called the uniform smoothing function, and is called the Picard smoothing function. The four functions belong to the class of the Chen-Mangasarian smoothing functions.
Many nonsmooth functions can be reformulated by using the plus function. We list some of them as follows:
So we can define a smoothing function for the above nonsmooth functions by a smoothing function of .
From Theorem and Corollary in [20], when is locally Lipschitz continuous at , the subdifferential associated with a smoothing functionis nonempty and bounded, and , where “con” denotes the convex hull. In [20, 23], it is shown that many smoothing functions satisfy the gradient consistencywhich is an important property of the smoothing methods and guarantees the convergence of smoothing methods with adaptive updating schemes of smoothing parameters to a stationary point of the original problem.
2.3. Pseudoconvex Function
Pseudoconvex function is a class of functions, which may be nonsmooth or nonconvex, but brings us the opportunity to find the optimal solutions.
Definition 2 (see [25]). Let be a nonempty convex subset of . A function is said to be pseudoconvex on if, for any , one hasMany nonconvex functions in application are pseudoconvex, such as the Butterworth filter function, fraction function, and density function. Of particular interest in this paper is the fact that the Gaussian function with , , is pseudoconvex on .
2.4. Project Operator
Let be a closed convex subset of . Then the projection operator to at is defined by and satisfies the following inequalities:(i) Suppose with ; then can be expressed by (ii) Suppose with of full row rank and ; then .
3. Optimization Problems in MDF Continuous Hot-Pressing Process
In this section, we will give the optimization model considered in this paper. First, by the optimization and support vector machine theories, we show the optimization models for building up the relationships in MDF continuous hot-pressing process. Then, another optimization model for optimizing the parameters in the MDF continuous hot-pressing process is obtained. Thus, we express these two kinds of problems into a uniform formulation, which is the optimization problem considered in Section 3.
3.1. Optimization Problem for Building up the Models of MDF Continuous Hot-Pressing Process
Denote and as two sets, where is the attribute vector on behalf of the hot-pressing plate properties; indicates the values of the qualities of hot-pressing plate. Let be the training data set of hot-pressing process, where obeys the unknown distribution and is IID (independent and identically distributed). Based on the support vector machine theory and the training data set, we would like to find a nonlinear function such that it approximates the training data set as much as possible.
With the kernel function and the Huber loss function, from the theory in [21], the dual optimization problem of the primal optimization problem of SVM is given asDenote and as the optimal solutions of (12). Then the regression function can be expressed bywhere can be calculated by one of the following two methods:
Denote in (12); then (12) can be reformulated aswhere , , , and with . For the optimal solution of (15), we call a support vector if . The optimal solution of (15) solved by the original methods often has almost 100% support vectors, which increases the complexity of the relation models largely. Thus, we introduce the problem deduced by (15); that is,with . In (43), is often called the regulation term, which is used to control the number of its support vectors.
Define the penalty function and then . From [19, Proposition ], is an optimal solution of (43) if and only if it is an optimal solution of the following problem:where . Thus, we can build up the relation models of hot-pressing process by solving problem (18).
3.2. Optimization Problem for Optimizing the Parameters in MDF Hot-Pressing Process
Based on the relationships built up in Section 3.1, we focus on the modulus of rupture (MOR), modulus of elasticity (MOE), and internal bonding strength (IBS) of hot-pressing plate by optimizing the process parameters and slab attributes. Suppose the regression functions of MOR, MOS, and IBS with respect to some relative parameters based on the Gaussian radial basis function kernel areand based on the linear polynomial kernel arewhere are the variables in the data set for regression, and indicates the independent variable with hot-pressing temperature , hot-pressing pressure , hot-pressing time , and moisture content .
The regression functions in (19) satisfy the following two properties.(i) is continuously differentiable on , .(ii) and are not convex, but is pseudoconvex on , .
And the regression functions in (20) satisfy the following two properties.(i) is continuously differentiable on , .(ii) and are convex on , .
From the physical significance of MOR, MOS, and IBS, we suppose that the larger the numbers of MOR, MOS, and IBS, the better the quality of hot-pressing plate. In order to adopt the different demand on the indexes of the hot-pressing plate in different applications, we consider the following two cases in this part.
First, we focus on maximizing a single performance index of the hot-pressing plate when the other two performance indexes are within the certain areas. If we want to optimize the IBS, the corresponding optimization model for this case can be expressed bywhere indicate the upper bounds of hot-pressing temperature, hot-pressing pressure, hot-pressing time, and moisture content, and , are the feasible regions of MOR and MOS, respectively.
In order to let problem (21) be solved effectively, we let the objective function be with the Gaussian radial basis function, and the regression functions and in the constraints are with the linear polynomial kernel. Then, (21) iswhich is a pseudoconvex optimization problem with convex constraints.
Second, we would like to optimize the MOR, MOS, and IBS synthetically. For this demand, we consider the following optimization model:where indicate the importance of MOR, MOS, and IBS, are with the same meaning as in (21), and , , and are the expected values of MOR, MOS, and IBS. In particular, if MOR, MOS, and IBS are with the same importance in the quality of the hot-pressing plate, we can let , and we can let , , and , if the importance of MOR, MOS, and IBS is strictly monotone decreasing. Similar to the kernel functions in problem (21), we let , , and in problem (23) be with the Gaussian radial basis function, which means that problem (23) is also a pseudoconvex optimization problem with convex constraints.
3.3. General Model
Based on analysis in Sections 3.1 and 3.2, we consider the following minimization problem in this paper:where , with , is continuously differentiable and pseudoconvex on , and () is continuously differentiable and convex on .
On the one hand, when , , , and and are defined as in (18), then problem (24) without reduces to problem (18). On the other hand, if we letthen problem (24) reduces to problem (22). Similar reformulation can be done for problem (23) by (24).
Therefore, problem (24) considered in this paper includes the optimization models for building up the relationships and optimizing the relative parameters in MDF continuous hot-pressing process.
In what follows, we denote as the feasible region of (24); that is, and is the optimal solution set of (24).
4. Main Results
4.1. Proposed Neural Network
In this subsection, we propose a one-layer recurrent neural network for solving problem (24), where we combine the penalty function and projection methods to solve the constraints and use the smoothing techniques to overcome the nonsmoothness of the objective function and penalty function.
Define the penalty function Then .
From the smoothing functions in (25) for the plus function, we define the smoothing function of as where
Form the results in [23], owns the following properties.
Lemma 3 (see [23]). (i) For any , is continuously differentiable, and is also continuously differentiable for any fixed ;
(ii) , , ;
(iii) , , ;
(iv) is convex for any fixed , and .
Then, has the following properties.
Lemma 4. is a smoothing function and satisfies the following: (i) is convex for any fixed ;(ii);(iii), , .
Next, by the smoothing function for the absolute value function we define
Since , owns all properties in Lemma 8 and the following results hold.
Lemma 5. is a smoothing function in (24) with the following properties: (i) is pseudoconvex for any fixed ;(ii);(iii), , .
From the projected gradient method and the viscosity regularization method, we introduce the following neural network to solve (24):where and with .
By the definitions for , (34) can be expressed aswhere
To implement (34) by circuits, we can use the reformulated form of (34) as follows:Equation (34) can be seen as a network with three input and three output variables that are , , and . A simple block structure of the proposed network (37) implemented by circuits is presented in Figure 1.
4.2. Theoretical Analysis
In this subsection, we study some necessary dynamical and optimality properties of proposed network (34) for solving (24).
The global existence of the solutions of (34) is a necessary condition for its usability in optimization. With an initial point , the solution of (34) is global existent. Moreover, the uniqueness of the solution of (34) with is proved under some conditions. The proposed network (34) can be implemented in circuits and mathematical software. Then, the feasibility and optimality of network (34) for optimization problem (24) are proved theoretically.
We call with a solution of (34) if is absolutely continuous on and satisfies (34) everywhere.
Proposition 6. For any initial point , there is a global solution of (34) defined on and it satisfies
Proof. Since the right hand function in network (34) is continuous about and , there are and an absolute continuous function such that satisfies (34) for all .
Denote . Then, (34) can be rewritten asA simple integration procedure of the above equation giveswhich can be rewritten asBy , , and , we obtain that , .
By the boundedness of and the extension theory, this solution of (34) can be extended. Thus, the solution of (34) with initial point is globally existent. Similarly, we can obtain that
Some Lipschitz condition is often used to guarantee the uniqueness of the solution of a neural network. In what follows, we give a sufficient condition to ensure the uniqueness of the solution of (34) with initial point .
Proposition 7. For any initial point , if and are locally Lipschitz continuous for any fixed , then the solution of neural network (34) is unique.
Proof. By Proposition 6, the solutions of (34) with initial point exist globally and satisfy , . Suppose that there exist two solutions and of (34) with initial point , and suppose there exists such that . By the boundedness of , there is such that and , .
Then, there is such thatDifferentiating along the two solutions of (34), by the Lipschitz continuity of and (43), we have Applying Gronwall’s inequality into the integration of the above inequality, it gives , , which leads to a contradiction. Therefore, the solution of (34) with initial point is unique.
Lyapunov method is employed to analyze the performance of (34). Here, we introduce the following two Lyapunov energy functions: The above two Lyapunov functions satisfy the following estimations along the solutions of (34).
Lemma 8. (i) The derivative of along the solution of (34) can be calculated by (ii) The derivative of along the solution of (34) can be calculated by
Proof. (i) Differentiating along the solutions of (34), we have Equation (34) can be rewritten asLetting and , using (49) and (10), we have which follows the fact thatCombining (48) and (51), we get that From Lemmas 4 and 5, and , we obtain the estimation in (i).
(ii) Differentiating along the solutions of (34), we haveLetting and , using (10) and (49), we havewhich can be rewritten asCombining (53) and (55), we getSimilar to the analysis in (i), we obtain the estimation in (ii).
Next, we prove the efficiency of proposed network (34) for solving optimization problem (24), where the convergence feasibility of the proposed network is a basic property.
Theorem 9. For initial point , any solution of (34) satisfies
Proof. Denote as a global solution of (34) with initial point .
From , and Lemma 8, is nonincreasing along the solution of (34). Using , , we confirm that From , and , we have which implies In what follows, we will prove that . Arguing by contradiction, we assume that By , , we have , which follows the fact that there is such thatFrom Lemma 4, we obtainwhich implies that there exists such that Since is bounded and , , by Lemma 5, there is such that Then, which implies that there is such thatFrom Lemma 8 and the convexity of for any fixed , we obtain thatBy (62), (67), and (68), we obtain Integrating the above inequality from to , we have Thus, which leads to a contradiction with for all and . Therefore,which guarantees that
The following theorem indicates that any accumulation point of the solutions of (34) is just an optimal solution of (34).
Theorem 10. For initial point , any solution of (34) is convergent to the optimal solution set ; that is,
Proof. From (63), (68), and , , we haveby , which can be rewritten asDenote Owning to the continuity of on , and are closed and open in , respectively.
Case 1. In this case, we assume that there exists such that , .
From the definition on and the pseudoconvexity of on , we have which implies By Theorem 9, we confirm that Case 2. In this case, we assume that there exists such that , , which means thatThen, Since exists and , we obtain thatSuppose Then, there is such that Then, from (76), we have Integrating the above inequality from to , we have Let in the above inequality; then we have which leads to a contraction with the boundedness from below on . Thus, which follows the fact that there is an increasing sequence such that andBy and which is bounded, there are and a subsequence of (denoted as ) such that and . By Lemma 5, there is such that From the pseudoconvexity of and the above inequality, we have By Theorem 9, we have . Therefore, , which implies that Combining the above results with (83), we conclude that .
Case 3. In this case, we assume that both and are unbounded.
For , similar to the analysis in Case , we haveFor , define . Then and . By the unboundedness of and , we haveFrom (81) and the continuity of on , we have By , (94), and (95), we have which gives From the definition of and the above result, we haveTherefore, from (94) and (99), we obtain
5. Numerical Experiments
In this section, we test the proposed neural network (34) for solving problem (24), which is brought forward from the MDF continuous hot-pressing process. Based on the existing data set, we use the established theories and proposed neural network (34) to build the relationships between the main qualities of the hot-pressing plate and some relative technology parameters from optimization problem (18). Then, based on optimization problem (22), we will use proposed network (34) to solve the optimal values of the technology parameters in hot-pressing system for optimizing the qualities of the hot-pressing plate. All these numerical experiments validate the good performance of the proposed network in this paper.
The numerical testing was carried out on a Lenovo PC (3.00 GHz, 2.00 GB of RAM) with the use of Matlab 7.4. And we use ode23 to realize the neural network (34) in Matlab.
5.1. Construction Relation Models in MDF Continuous Hot-Pressing Process
In this part, by considered optimization problem (18) and network (34), we build the relation model which takes the hot-pressing temperature (TE), hot-pressing pressure (PR), hot-pressing time (TI), and moisture content (MC) of slab as the argument variables and the MOR, MOE, and IBS indexes of MDF as the dependent variables. The numerical results show the good fitting of the built models for the data set, where the data set is given in Table 4.
In order to use the data in Table 4, we first normalize them into . And we use the mean square error (MSE), degree of fitting (DF), and sufficient evaluation (SE) to evaluate the numerical results, where where and indicate the actual value and predicted value and and are defined as in (18). The smaller the MSE and the SE and the closer the DF to , the better the regression result. Moreover, the SE function is the objective function in (15).
Based on the Gaussian radial basis function kernel, the values of the initial parameters in problem (18) are given in Table 1. With a random initial point , the numerical results with respect to the obtained solution are also listed in Table 1. Figures 2–4 illustrate the fitting effect of the MOR, MOE, and IBS values by using the proposed network (34) for solving problem (18). And the parameters of the regression functions in (19) are shown in Table 5.
Moreover, the regression functions based on the linear polynomial kernel in (20) are also calculated by network (34), where the parameters are also shown in Table 5.
5.2. Optimization of Parameters in MDF Continuous Hot-Pressing Process
In this subsection, we consider two classes of parameter optimizing problems, one aims at optimizing one performance index of MDF, and the other is for comprehensively optimizing the three performance indexes of MDF. Based on the obtained relation models for a particular slab and hot-pressing system, we give a suggestion on the setting of hot-pressing pressure, hot-pressing temperature, hot-pressing time, and moisture content of slab to let the MDF meet the given requirements. We refer to the current standard of MDF indoor plate in China (GB/T11718-1999), which is given in Table 2.
5.2.1. Case 1
Case 1. When the MOR and MOE of the hot-pressing plate are in the certain regions, we would like to maximize the IBS by control of the hot-pressing pressure, hot-pressing temperature, hot-pressing time, and moisture content of slab.
By the information in Table 2, we let , , , and and choose , , , , , and in optimization problem (21).
Then, , and we can let
By the proposed network (34) for solving (21), we obtain the optimal solution This means that when we let the hot-pressing temperature be 159.4725°C, hot-pressing pressure be 3.1022 Mpa, hot-pressing time be min, and moisture content of slab be , we can maximize the IBS of the hot-pressing plate and let the MOR and MOE of it be in the certain regions, where the three performance indexes are shown in Table 3. The convergence of the MOR regression function , MOE regression function , and IBS regression function along the solution of (34) are plotted in Figures 5–7.
5.2.2. Case 2
Case 2. We would like to maximize the MOR, MOE, and IBS synthetically by control of the hot-pressing pressure, hot-pressing temperature, hot-pressing time, and moisture content of slab.
In this case, we use optimization problem (23) with , , , , , , , (Mpa), (Mpa), and (Mpa).
Using proposed network (34) to solve (23), we obtain the optimal solution The convergence of along the solution of (23) is plotted in Figure 8, where the convergence of MOR regression function , MOE regression function , and IBS regression function is shown in Figures 9–11.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This work is supported by the Fundamental Research Funds for the Central Universities (2572014AB03) and the National Natural Science Foundation of China (31370565).