Abstract
A wavelet neural network with time delay is proposed based on nonlinear autoregressive model with exogenous inputs (NARMAX) model, and the sensitivity method is applied in the selection of network inputs. The inclusion of delayed system information improves the network’s capability of representing the dynamic changes of time-varying systems. The implement of sensitivity analysis reduces the dimension of input as well as the dimension of networks, thus improving its generalization ability. The time delay wavelet neural network was implemented to real-time ship motion prediction, simulations are conducted based on the measured data of vessel “YUKUN,” and the results demonstrate that the feasibility of the proposed method.
1. Introduction
Ship motion dynamics is related greatly to the navigational conditions such as loading and draft; the changes of dynamics also are influenced by environmental disturbances such as wind, wave, and current. Therefore, the identification and control of ship motion is complex and is being a challenge in the area of ship engineering [1]. Furthermore, the accurate identification and prediction are important issues in ship and its controller design. To overcome the problems of accuracy and adaptiveness in applying the custom methods, more and more current studies incorporate the booming artificial intelligent techniques such as neural networks and fuzzy inference [2].
As a kind of data-driven approach [3–5], neural network gained much popularity in recent years attributed to its merits such as nonlinearity and adaptivity. Among various kinds of neural networks, backpropagation (BP) network is the most widely used network type. But the BP network has its drawbacks such as slow convergence speed and existence of phenomenon of local minima, which limited its practical engineering applications. Various types of neural networks and their learning algorithms have been proposed. Wavelet neural network (WNN), which was proposed in late 20th centuries, is a kind of local response type neural network and attracts many research interests in areas of intelligent computation. It makes use of wavelet function as active function instead of sigmoid function in BP network and establishes the connection between the wavelet transform and network parameters by using affine connection [6]. Wavelet has characteristics of time-frequency local and zoom ability, and neural network has characteristics of strong abilities of self-learning, robustness, and generalization. The wavelet neural networks take both advantages of wavelet transform and neural network and realizes wavelet transform by adjusting wavelet neurons adaptively. Wavelet network has been demonstrated to have good function approximation and pattern recognition capabilities, which have been utilized successfully in various areas such as adaptive control, signal filtering, and fault detection [7].
Various measures have been taken to improve the approximation and prediction ability of neural networks such as raw data processing [8]. In order to represent the dynamic characteristics of time delay system more efficiently, we improve wavelet network in this study by introducing the history information of system into the network input layer, construct the wavelet network with time delays, and realize the model of nonlinear autoregressive moving average with exogenous inputs (NARMAX) [9]. The neural network with time delay possesses the same network structure as the NARMAX model and both of them take consideration of history information of system dynamics, and NARMAX has been proved to be able to represent input-output mapping of time-varying dynamics; therefore, the proposed method has the inertia ability to represent time-varying system dynamics. Sensitivity analysis method is applied for deciding the inputs of the mode to reduce the number of inputs and hence the dimension of neural network consequently, which is an important issue in improving the generalization ability of the network. To demonstrate the efficiency of the proposed wavelet network model, the measured ship motion data of motor vessel “YUKUN” is implemented for simulation experiment.
2. NARMAX Prediction Model Based on Time Delay WNN
2.1. Network Structure of Time Delay WNN
There are three layers in conventional neural network including input layer, hidden layer, and output layer. In the proposed wavelet neural network, the current input information is conveyed to the input layer of wavelet neural network together with history information of system, and the information is conducted and the results are conveyed to the output layer. The network processes the current and historical information of system simultaneously. After the learning and fitting processes of network, the acquired network is then applied for prediction of time delay system [10, 11] based on the NARMAX model. The structure of the proposed time delay WNN is depicted in Figure 1.
NARMAX model is widely used in system identification and prediction [12, 13]: where x and y are system input and output and ny and nu are the order of system output and input, respectively. It can be noted in (1) that the multidimension inputs of NARMAX model include the past ny system outputs and nu system inputs.
Equation (1) is the identification model of the mapping between the current system with respect to the history system inputs and outputs. When the identified model is utilized for prediction, we can replace the t in the equation with , and we get
If we need to give multistep predictions, for example, 2-step-ahead prediction, we need to replace the t in the equation with t+2 likewise: where is the predicted value of . In multistep ahead prediction process, as the value of is unknown in time of t, in the cascade prediction method, the predicted value by (2) is used to substitute required in (3) to realize the 2-step-ahead prediction.
The multistep prediction process can be realized likewise. But in this cascade prediction method, the prediction accuracy of multistep prediction based much on the accuracy of the previous predicted results and all the prediction error in the previous steps are brought to the final prediction result. So, another kind of prediction, which is referred to as direct prediction, is adopted in this paper. As direct prediction, process the prediction of different steps ahead individually and avoid the accumulation of prediction error consequently.
2.2. Learning Algorithm of Wavelet Neural Network
2.2.1. Initialization of WNN Parameters
Let the number of WNN inputs be M and the number of hidden units. In WNN, the most conventionally used wavelet active functions in hidden layer are Morlet wavelet, Harrab wavelet, and Gaussian wavelet. In this study, we make use of Morlet wavelet as the active function of WNN:
Time delay wavelets have merits such as high resolution ratio, strong robustness, and efficient computation ability as well as low error rate [14]. The network output is computed as follows: where ak and bk are scale factor and translation factor of basis function, respectively. And wk is the connection weight between the kth hidden wavelet neuron and the output neuron.
2.2.2. Adjustment of WNN Parameters
The cost function of WNN learning is expressed in where is the desired output, is the actual output, tl is the sample input, and l is the number of samples. Here, let
The gradient messages of wk, ak, and bk are expressed hereunder:
Based on the gradient message, the parameters are adjusted by minimizing the cost function. The iterations of adjustment should take into consideration both accuracy and training speed.
2.3. Selection of Network Inputs Based on Sensitivity Analysis
The method of sensitivity analysis is usually used in evaluating the influences of the changes to the model output, and it plays an important role for model construction, model simplification, and thus the generalization ability of model. It has been widely used in areas such as engineering, economics, and sociology [15, 16]. Rhee and Kim analyze the relationship between parameter sensitivity and identification accuracy [17].
For the NARMAX model based on neural network, we need to establish suitable inputs for the model [18]. If there are too few inputs, the network cannot represent the input-output mapping of system with sufficient accuracy; if there are too many inputs, the network dimension will increase consequently. Both of them will deteriorate the generalization capability of the network. As there are no optimal input variables so far for ship motion prediction, it is appropriate to evaluate the importance of individual input and decide which inputs should be taken into the input layer of network.
The input-output derivatives are useful indices for evaluating the network’s input to the output. There are two kinds of conclusions which can be drawn from the values of derivatives. One is the influences of tiny changes of input to the output; the other is the contribution of each input to the overall output.
Morlet wavelet like (4) is implemented in this study and served as the active function of hidden neurons. According to (10), we can calculate the derivatives of output to each input and evaluate the influence of the input to the output. Negative value of the derivative shows that, with the increase of the input, the output will decrease; and the positive value of the derivative shows that the increase of the input will cause the increase of output consequently.
Based on the calculation of the derivatives, the influence of input to the output can be derived accordingly. The influence can be measured by index of sum of square derivatives (SSD): where where Sj is the derivative of output with respect to input , Ihj is the response of the hth hidden neuron, and who and wih are the connection weights between the output layer to the hth hidden neuron and the hth hidden neuron to the ith input, respectively.
The values of SSD corresponding to each input can be used to evaluate the contribution of each input to the output. It can be noticed that the threshold of SSD for individual input can influence the selection greatly. To improve the robustness of the selection, we set up the threshold for the sum of SSD. That is, we sort the influence of inputs according to corresponding values of SSD in inverse order. The selection begins with the smallest SSD and so on till the iSth SSD:
The first to the iSth neurons are selected based on the selection of corresponding values of SSD. By this deletion process, the inputs with correlation little to the output will be deleted from network. The deletion can lead to a compact network structure and hence improve the generalization ability of network. In implementing the aforesaid pruning method, we set a larger number of inputs initially and prune those contributing less to the output.
To improve the generalization ability, we need a compact network structure. However, the dimension of network involves not only the number of inputs but also the number of hidden neurons. In deciding the number of hidden neurons, we make use of Akaike information criterion (AIC) to choose the optimal neuron number when the value of AIC reaches its minimum. AIC is an index for achieving network structure with optimal generalization ability, which considers both hidden neurons number and identification accuracy [19].
3. Simulation Experiment of Ship Motion Prediction
To validate the prediction capability of the proposed time delay wavelet neural network, we make use of the measurement data of motor vessel “YUKUN,” which is a training and scientific researching ship of Dalian Maritime University. The application of real measured data can also validate the antidisturbance ability because the measured data have taken the influences of wind, wave, and current.
The shipboard measuring system can measure the ship’s motion status precisely which includes the wind, wave, current, and the measurement noise. The main principals of “YUKUN” are shown in Table 1.
The measured data during sea voyage of “YUKUN” is adopted for the simulation experiment of ship motion prediction. The prediction model is NAMAX which is realized by time delay WNN, and the inputs are selected through sensitivity analysis method. The synchronously measured raw data include longitude, latitude, heading degrees, roll angle, pith angle, speed over ground, and speed over ground. To establish the prediction model, the contributions of inputs to the output are firstly evaluated by the sensitivity analysis and the inputs are selected as 6 variables: , , , , , and , where is the roll angle of ship motion. The sample interval of the measurement is 1s. So the k-step-ahead prediction means prediction of k seconds ahead. Simulation of 300 steps was conducted for each simulation.
Therefore, with the selected inputs and the desired output, the ship heading angle prediction model is established as
It can be noticed that the model is NARMAX which is realized by time delay WNN. In the simulation, the samples are presented to the WNN sequentially, which is encountered in practical applications like online ship motion prediction and control. And there are two stages of computation in each step. Firstly, the presented data are used to train the WNN and the network parameters are determined accordingly; once the model is established and the parameters are decided, the achieved network will be implemented for prediction. That is, with the trained model in (14), the ship motion of 1-step-ahead is achieved:
For multistep prediction, we use direct prediction instead of cascade prediction. For 2-step-ahead prediction, the input-output mapping is identified:
After the training process, the predictions can be made with the current available ship motion information at time of t:
Likewise, the method can be generalized to conditions of multistep prediction. For k-step-ahead prediction, the identification model and prediction model can be represented hereunder:
To evaluate the prediction ability for multistep prediction, 1-, 2-, and 3-step-ahead predictions were conducted, respectively, attributed to the nature of direct prediction where the more steps of prediction do not depend on the results of the previous less steps of prediction. The measured ship heading angle during the ship voyage is depicted in Figure 2.
It can be noticed from Figure 2 that, as influenced by wind, wave, and current when sailing at sea, the heading degree changes violently. As stated above, ship motion is a complex process with nonlinearity, time-varying dynamics, and uncertainty and hence is hard to predict accurately. Therefore, the measured ship motion data is utilized to validate the efficiency of the proposed prediction model. For 1-step-ahead prediction, the identification error curve is depicted in Figure 3.
The index root mean square error (RMSE) is adopted for evaluating the identification ability and the RMSE is 0.2019° for 1-step-ahead prediction. It is shown from Figure 3 that the identification error is small and averaged around zero. Based on the identification, the prediction was made and the predicted heading angles together with measured ones are shown in Figure 4.
It can be seen from Figure 4 that the predicted results coincide with the measured ones precisely. The RMSE of prediction is 0.3289° for 1-step-ahead prediction. The change of prediction error is illustrated in Figure 5.
For 2-step-ahead prediction, the same structure of time delay WNN is adopted and no further adjustments are made for the prediction model structure. The identification error for 2-step-ahead prediction is shown in Figure 6.
In the 300 steps of prediction, the identification RMSE is 0.2002° and the 2-step-ahead prediction result is shown in Figure 7.
The prediction RMSE is 0.3498° and the changes of prediction error are shown in Figure 8.
For 3-step-ahead prediction, the same WNN structure is adopted as 1-step- and 2-step-ahead prediction. The change of identification error is shown in Figure 9.
The identification of RMSE is 0.2313°. The prediction is conducted based on the identification result, and the prediction result is shown in Figure 10.
The prediction of RMSE is 0.3727°; the prediction errors are shown in Figure 11.
It can be noticed from simulation results that as the prediction horizon increases from 1 s to 3 s, the time delay WNN can generate prediction results with satisfactory accuracy. Prediction accuracy does not vary much with the increase of prediction horizon. This shows that the direct prediction avoids accumulation of prediction error and is an efficient method for multistep prediction. And the structure of time delay WNN remains unchanged for all the simulations, which means that the prediction method based on WNN possesses satisfactory robustness.
In the process of 300 steps of simulation, the 1-step-, 2-step-, and 3-step-ahead predictions cost time of 2.68 s, 2.80 s, and 2.77 s, respectively. The average processing time of 1 step is less than 0.01 s. This shows that the time delay WNN processes fast.
The simulation based on measured data demonstrates that the proposed method can generate multistep predictions with high accuracy attributed to the adaptive network structure and local response feature, and its processing speed is fast attributed to its compact network structure, which are important for its practical online applications such as online prediction and control.
4. Conclusions
A wavelet neural network with delayed system information is constructed and applied for online ship dynamics prediction. The sensitivity analysis is also applied to determine the inputs to the WNN, which improve the generalization ability of network. Simulations of ship motion prediction are conducted based on the real data measured during sea voyage at sea and the results demonstrate that the model has fast processing speed, satisfactory accuracy, antidisturbances ability, and robustness. Our future work will focus on the online adjustment of network dimension to adapt to the complex changes of ship dynamics. The adaptive wavelet neural network can be used for online ship motion status prediction and ship navigational safety evaluation and control.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grant no. 51279106), the Natural Science Found of Liaoning Province (Grant no. 2014025008), and Fundamental Research Funds for the Central Universities (Grant no. 3132014028).