Mathematical Problems in Engineering

Volume 2015, Article ID 362150, 11 pages

http://dx.doi.org/10.1155/2015/362150

## Model and Algorithm of BP Neural Network Based on Expanded Multichain Quantum Optimization

^{1}PLA University of Science and Technology, Nanjing 210007, China^{2}Bengbu Automobile NCO Academy, Bengbu 233011, China^{3}Logistics Academy, Beijing 100858, China

Received 30 August 2015; Accepted 15 October 2015

Academic Editor: Peter Dabnichki

Copyright © 2015 Baoyu Xu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

The model and algorithm of BP neural network optimized by expanded multichain quantum optimization algorithm with super parallel and ultra-high speed are proposed based on the analysis of the research status quo and defects of BP neural network to overcome the defects of overfitting, the random initial weights, and the oscillation of the fitting and generalization ability along with subtle changes of the network parameters. The method optimizes the structure of the neural network effectively and can overcome a series of problems existing in the BP neural network optimized by basic genetic algorithm such as slow convergence speed, premature convergence, and bad computational stability. The performance of the BP neural network controller is further improved. The simulation experimental results show that the model is with good stability, high precision of the extracted parameters, and good real-time performance and adaptability in the actual parameter extraction.

#### 1. Introduction

Artificial neural network (ANN) is a new information processing and computer system which is based on the modern neuroscience research and is formed by abstracting, simplifying, and simulating of biological structure. The features of ANN are as follows. ANN can fully approximate to any complex nonlinear relations; all quantitative or qualitative information keeps in storage equipotentially in each neuron of network so it is with very strong robustness and fault tolerance; ANN is a kind of system which can emulate and adapt to the unknown system and is able to deal with the quantitative and qualitative knowledge at the same time [1]. The main research of ANN is how to make computer simulate and realize the self-learning and mathematical thinking ability of human to mine the inner relation from limited samples. It is mainly through studying and storing the data relations, the inference rules, and probability distribution of known sample to deduce and reveal the potential information between variables in the unknown data sample [2].

BP neural network is currently the most popular neural network model in application [3]. BP algorithm was proposed by Rumelhart and Mcllelland in 1986 which well solved the weight adjustment problems of nonlinear continuous function in the field of multilayer feedforward neural network. It is a typical error back propagation algorithm [4, 5]. Selection of activation function, design of structure parameter, and improvement of network defect have been researched a lot since the emergence of the BP neural network. In 1973, Grossberg found sigmoid function is very similar to the work situation of biological neurons so he began to explore the relationship between the features of sigmoid function and the stability of the neural network. He prompted the function to become the most commonly used activation function of BP neural network. Since then many scholars have done improvements on the limitations of the sigmoid function. In order to solve the problem that BP algorithm can obtain larger gradient values in the whole domain, many scholars try to combine activation functions with different characteristics in different intervals together to set a larger derivative value at the needful position to make up for the inadequacy of single activation function [6]. In 1991, Kung and Hu [7] proposed FARM approximate simplified method which uses Frobenius norm according to the ideas of the global optimization and gradual optimization to delete the hidden layer units and determine the number of hidden layer nodes. Fahlman [8] found that if we adopt the sigmoid function as the activation function, the derivative of error of weight will become very small when the output value of BP network is close to 0 and 1. Salomon [9] created two BP neural networks with exactly the same initial parameters and structure and put forward a new method of network learning. The learning rate is increased and decreased, respectively, to update the weight of two networks and network is set with fast error drop as the starting point of the next update. The effect is obvious. Dan Foresee and Hagan [10] proposed BFGS quasi-Newton method which not only can avoid calculation of the second derivative but also keeps the advantage of fast convergence of the Newton algorithm. Riedmiller and Braun [11] proposed elastic BP (RPROP) method in 1993. RPROP method introduced probability of resilient update value to modify weights directly which can reduce influence of the network structure parameters in the whole learning process and avoid unforeseen gradient error convergence of fuzzy network performance.

At present, the research based on ANN to carry out quantitative structure-activity relationship research has gradually been applied to various fields. Wang et al. [12] utilized neural network to study the quantitative structure-activity relationship of angiotensin converting enzyme inhibitors. Zhao et al. [13] established the model of antitumor activity of emodin derivatives on the basis of neural network. Cui et al. [14] applied the principal component analysis and neural network method to study the quantitative structure-activity relationship of the nitrobenzene and its homologue. González-Díaz et al. [15] established some quantitative structure-activity relationship models of synthetic compounds by neural network and verified the reliability of the model. The model can be used to design new drugs. Prado-Prado et al. [16] studied quantitative structure-activity relationship of parasite drug resistance between different kinds of parasites through neural network. Ramírez-Galicia et al. [17] established quantitative structure-activity relationship model of amoeba drug resistance through multiple linear regression, stepwise regression analysis, and neural network. The result shows that the three-dimensional structure of the model is very important.

Although BP algorithm has become the most widely used artificial neural network, the BP neural network has the following defects.

It falls into local minimum value easily: the BP neural network learning algorithm based on gradient descent method is easy to fall into local extremum points and saddle point if error function is not strictly convex function and there are multiple points whose gradient is zero. Then, the network can not converge and is unable to get the optimal solution of the problem so the optimal network connection weights and threshold parameters can not be obtained [18].

The speed of error convergence is slow: convergence speed of BP neural network is decided by two aspects: one is learning rate and the other one is the size of derivative of related excitation function [19]. It is usually not easy to choose the size of the learning rate. If the learning rate is too large, the oscillation or even no convergence phenomenon will happen in the process of training. The learning rate should be a small positive number. If the learning rate is too small, the product of learning rate and negative gradient vector will become smaller. Then, the renewal speed of weights and thresholds will be affected. In addition, the size of derivative of the activation function also affects the rate of convergence. In the flat area of error curved surface, the gradient of error function is small to make update speed of weights and threshold of network slow. The network needs to pass through much iteration to be out of the flat area so the convergence speed of the network becomes slow.

Structure of network is not easy to determine: the determination of the structure of BP neural network usually refers to determination of the number of hidden layers and the number of neurons of hidden layer. Especially after Kolmogorov et al. prove the single hidden layer of BP neural network approximation theorem, how to select the number of neurons in single hidden layer neural network has always been number one of the hot and key problems. Generally, the number of neurons of input layer and output layer is easy to determine according to identification objects. The number of neurons of hidden layer is difficult to determine, and it directly affects topology structure and performance of BP neural network. It has been proved in theory that the three-layer BP neural network can approximate nonlinear functions of arbitrary precision which solves the problem of determination of the number of hidden layers in the BP neural network. If the number of the neurons of hidden layer is too small, the network can not meet the requirements of learning and the approximation performance; if the number of the neurons of hidden layer is too much, there will be adverse phenomenon in the network and it will make the hardware implementation and software calculation complicate at the same time. At present, there is no unified and complete determination theoretical framework to determine the structure of the BP neural network. The experience or grope through lots of experiments is the usual manner to estimate and adjust the structure of neural network.

In order to further improve the efficiency of the BP neural network and overcome the shortage of it, a lot of research has been conducted. Sun et al. established improved BP neural network prediction model and quantitatively researched related parameters [20]. The accuracy of the model was improved to some extent. Xiao et al. proposed BP neural network with rough set for short term load forecasting. The accuracy of prediction of BP neural network was further improved [21].

Genetic algorithm imitates evolutionary and genetic rule of biology and is a mathematical algorithm which can solve problems to find the global optimization. Due to the strong macroscopic search ability and good global optimization performance of genetic algorithm, many scholars try to use genetic algorithm to optimize the connection weights, structure, learning rules, and so forth of BP neural network. Xiao et al. [22] applied genetic algorithm to constitute GA-ANN method to optimize the optimization problems in complex engineering. The method not only takes advantage of the nonlinear mapping, network reasoning, and predicting function of neural network but also uses the characteristics of global optimization of genetic algorithm. It can be widely applied to many complex engineering problems whose objective function is difficult to express by the form of explicit function of decision variables. Yang et al. [23] applied genetic algorithm to select parameters which overcame the restriction of symmetric weight matrix of traditional fluid neural network and broadened the application fields of this intelligent exploration method. Ge [24] applied genetic algorithm to optimize the controller parameters of structure of the neural network and applied the controller to control object with pure lag. The experiment proved that the control system optimized by genetic algorithm was with good static performance and dynamic performance. Li et al. [25] combined genetic algorithm with traditional DBD algorithm to propose a new algorithm to optimize the BP neural network, making network with large scale be able to converge fast and be out of the trap of local minimum. This algorithm is less sensitive to the selection of network parameters and obtains good effect in the application of missile comprehensive test.

In order to further overcome the shortage of the model based on genetic algorithm to optimize the BP neural network, the model and algorithm of BP neural network based on expanded multichain quantum optimization are proposed. The structure of neural network is effectively optimized. The model can overcome a series of problems of basic genetic algorithm, such as the slow convergence speed, premature convergence, and bad computational stability to further improve the performance of the BP neural network controller.

#### 2. BP Neural Network Model and Algorithm

The main idea of BP algorithm is to divide learning process into two stages, that is, positive communication of signal and back propagation of error. In the stage of positive communication, input information is from the input layer to output layer through the hidden layer [26]. The output signal forms on the output side. The weights of network are fixed in the process of the signal transmission forward. The state of neurons of each layer only affects state of neurons of the next layer. If the desired output can not be achieved in the output layer, then the error signal will back propagate. In the back propagation stage, the error signal which failed to meet the accuracy requirement spreads forward step by step and the error is shared by all units of each layer. The connection weights are adjusted dynamically according to the error signal. The weight value between neurons keeps correcting through the cycle of forward and back adjustment. The learning stops when the error of the output signal meets the requirement of precision [27].

##### 2.1. Topology Structure of BP Neutral Network

The simplest BP neural network is with three layers as is shown in Figure 1. It includes input layer, hidden layer, and output layer. The number of nodes in the input layer is equal to the dimension of input vector . The number of nodes in the output layer is equal to the type of output module . The number of nodes in the hidden layer is associated with specific application which is usually selected by test.