- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Journal of Applied Mathematics
Volume 2013 (2013), Article ID 983051, 9 pages
Incomplete Phase Space Reconstruction Method Based on Subspace Adaptive Evolution Approximation
1School of Electrical and Information Engineering, Chongqing University of Science and Technology, Chongqing 401331, China
2School of Electronic Engineering, Xi'an Shiyou University, Xi’an 710065, China
Received 19 July 2013; Accepted 19 September 2013
Academic Editor: Baocang Ding
Copyright © 2013 Tai-fu Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The chaotic time series can be expanded to the multidimensional space by phase space reconstruction, in order to reconstruct the dynamic characteristics of the original system. It is difficult to obtain complete phase space for chaotic time series, as a result of the inconsistency of phase space reconstruction. This paper presents an idea of subspace approximation. The chaotic time series prediction based on the phase space reconstruction can be considered as the subspace approximation problem in different neighborhood at different time. The common static neural network approximation is suitable for a trained neighborhood, but it cannot ensure its generalization performance in other untrained neighborhood. The subspace approximation of neural network based on the nonlinear extended Kalman filtering (EKF) is a dynamic evolution approximation from one neighborhood to another. Therefore, in view of incomplete phase space, due to the chaos phase space reconstruction, we put forward subspace adaptive evolution approximation method based on nonlinear Kalman filtering. This method is verified by multiple sets of wind speed prediction experiments in Wulong city, and the results demonstrate that it possesses higher chaotic prediction accuracy.
In recent years, industrial disasters and accidents occurred frequently, the meteorological and hydrological conditions were complicated and changeable, and financial markets fluctuated drastically. These phenomena often contain chaotic characteristics [1, 2], and prediction  for these phenomena is imminent. For a long time, there was no scientific tool for handling this issue, because the changing mechanisms of characteristics in these phenomena were not understood very well. Hence, aiming at the chaotic characteristics, some scholars worked with structures and made a lot of new researches on the prediction of chaotic time series [4–8].
To study and deal with the measurement data of chaotic system, Kennel et al. presented the reconstruction method of phase space system. Two parameters, the embedding dimension and delay time , needed to be determined before the phase space reconstruction [9, 10]. At present, time delay selection methods that are commonly used in the chaotic short-term prediction mainly include autocorrelation method , mutual information method , and singular value fraction method . Calculating methods of embedding dimension mainly include saturated correlation dimension , false nearest neighbors method , and Cao's method . Hu and Chen put forward the C-C method , which can simultaneously estimate the delay time and embedding dimension with the correlation integral. Autocorrelation method extracts only linear correlation degree between time series, which is hard to be applied to high-dimensional chaos system and nonlinear dynamical system. Mutual information method, which can determine the optimal delay time by calculating the first minimal value of mutual information function, is a nonlinear analysis method, but it cannot avoid massive calculation and cannot satisfy the requirement of complicated space division. It is difficult to determine the threshold of the singular value, because the singular value fraction method is largely affected by noise. When the embedded dimension with the saturated correlation dimension is calculated, the main question is to choose the different neighborhood radius. The radius selection has certain randomness, and the result will be in large deviation with improper choice, because of the influence of noise in the data and excessive concentration of the data. The determination of threshold has very strong subjectivity when we use false nearest neighbors method to determine the embedding dimension. There is no objective standard to determine the threshold value, especially for the experimental data, which may get a wrong result. Cao's method, an improved false nearest neighbors method, can effectively distinguish random signals and deterministic signals, and embedding dimension can be obtained through a less amount of data. C-C method is based on the statistical theory, so cannot be precisely determined.
Researches have showed that different phase space reconstruction methods get different and . Moreover, the same chaotic time series with the same kind of method in different times may get different and . There is no phase space reconstruction that can obtain complete and independent phase space. After phase space reconstruction, prediction model is often established through the functional approximation method.
The prediction model based on phase space reconstruction has been used to adopt the functional approximation method based on the neural network [18–21], which has strong nonlinear fitting capability and can approximate any complex nonlinear relationships. However, since neural network is only suitable for approximation of a deterministic system, it is difficult to guarantee the time-varying system performance and ensure its generalization performance in other untrained neighborhood. Meanwhile, the prediction effect of neural network is not good, because the chaotic time series is a complex nonlinear uncertain system.
In this study, we introduce Kalman filtering to neural network model , inspired by Kalman iteration and Bucy and Sunahara’s nonlinear extended Kalman filtering theory . The subspace approximation of neural network based on the nonlinear extended Kalman filtering (EKF) has a function which is dynamic evolution approximation from one neighborhood to another. Therefore, we can constitute a phase space by choosing a kind of phase space reconstruction method, and the space may be incomplete, not separate, and can be seen as a subspace of the ideal phase space. On this basis, we put forward adaptive neural network model based on nonlinear Kalman filtering and finally realize the subspace approximation of dynamic evolution system. In addition, we simulate wind speed series in Wulong city using the proposed method. By comparing with BP neural network prediction model, the results show that our method possesses higher prediction accuracy.
The paper is organized as follows. Section 2 discusses about the subspace approximation of phase space reconstruction. In Section 3, we describe the neural network model based on nonlinear Kalman filtering. Section 4 uses practical examples and series tests to verify the proposed method, while Section 5 contains the conclusions of the present work.
2. Subspace Approximation of Phase Space Reconstruction
Reconstructing phase space by chaos theory needs to identify the chaos of time series. Single variable time series can be reconstructed into a phase space by Takens’ embedding theorem in phase space reconstruction [24, 25]; that is, the original dynamical system can be restored in the sense of topological equivalence as long as the embedding dimension is sufficiently high. For the observed time series , after time delay reconstruction by Takens embedding theorem, it will receive a set of space vector After phase space reconstruction, the data space is Accordingly, we acquire where is a single-valued function. Then, we have However, it cannot be really obtained as the data are often limited. Hence, can only be constituted by limited measurement data, making sufficiently approximate to , consequently we can get a nonlinear prediction model.
This paper employs the neural network to predict chaotic series. However, the neural network cannot readily handle the inconsistency of the phase space reconstruction because of uncertain nonlinear chaotic time series. Therefore, it is crucial to adaptively construct subspace to approximate chaotic series through the incomplete phase space. The feature of adaptive subspace approximation is that it can add new data in real time and forget old data in the process of training. Consequently, weights and thresholds of the neural network are continuously modified to realize the dynamic evolution modeling.
3. Neural Network Model Based on Nonlinear Kalman Filtering
Kalman filtering has good adaptability. It can dynamically update and forecast the system information in real time with limited data. However, it cannot be readily used for complicated nonlinear model. Meanwhile, the extended Kalman filtering (EKF) is a kind of effective method to handle nonlinear filtering.
The mathematical model of EKF is as follows: where and are independent, zero mean, and Gaussian random processes with covariance matrices and , respectively. The statistical properties are as follows:
EKF spreads nonlinear functions and to Taylor series around filtering value and predicted value , respectively, only retaining the first-order information. Hence, the linearization model of the nonlinear system is obtained, and then we can obtain the EKF formula in nonlinear system by basic equations of Kalman filtering.
Given a forward network with layers, the numbers of neurons in each layer are . Suppose that input layer is the first layer and output layer is the layer. The weights of the layer neurons are . In order to convert the calculation of connection weights in the above problem into filter recursive estimation form, we let all of the network weights constitute the state vector where state vector consists of all of the weights according to the linear array, and its dimension is as follows: Then the state equation and measurement equation of the system can be expressed as where is the expected output, is the input vector, and is the actual output.
The measurement noise is assumed to be additive, white, and Gaussian, with zero mean and with covariance matrix defined by Suppose that the output of the node for the layer in the iteration is From (10) and (12), we have Assume that Accordingly, the measurement equation may also be expressed as The Jacobian matrix of the function is described by Similarly, all thresholds of the network constitute the state vector where the dimension is Suppose that and are both state variable; that is, the state vector composed of weights and thresholds is described by Kalman filtering algorithm on training weights and thresholds of the neural network is as in Table 1.
4. Simulation Examples
4.1. Determining of Embedding Dimension and Delay Time
One of the most popular chaos logistic mapper is selected as the study object. Logistic equation is
The related time series are produced according to (20). It is a chaotic system when . Assume that initial value of series is 0.1, and 4000 points are calculated. The first 1000 points are eliminated as transition phenomenon, leaving the remaining 3000 points to reconstruct phase space. Before the phase space reconstruction, we determine the embedding dimension and delay time . A comparison among several methods is present in Table 2.
Obviously, the optimal embedding dimension and delay time are generally different by different methods of phase space reconstruction.
In order to verify the fact that data at different time will obtain different embedding dimension and delay time with the same phase space reconstruction method, we have the following experiment.
The remaining 3000 points () are divided into five parts, with time intervals , , , , and , respectively. Embedding dimension and delay time are present in Table 3 by C-C method.
Apparently, the data during different time periods will acquire different embedding dimension and delay time by using the same phase space reconstruction method.
4.2. Wind Speed Chaotic Series Forecasting Simulation
Analysis about the chaotic characteristics of wind speed in the process of wind power generation has been presented in a related article . We record one of the wind speed data every 10 minutes, and 150 groups of wind speed data in Wulong city are used to simulate experiments in our study. We obtain the corresponding and by different phase space reconstruction methods, as shown in Table 4.
Various combinations are present in Table 5.
Wind speed prediction [27, 28] of chaotic time series about neural network model usually extracts phase space reference points as the BP neural network training samples on the basis of phase space reconstruction. We establish the neural network model based on nonlinear Kalman filtering, including two parts: predict wind speed and constantly modify weights and thresholds of the neural network by Kalman recursion. In this paper, BPNN model in the same structure is employed to forecast wind speed time series, in order to illustrate the validity of EKFNN on predicting the chaotic time series. The same 150 groups of wind speed data are used to simulate experiments. The predicted curves and error curves are shown in Figures 1, 2, 3, 4, and 5.
Comparisons among several models in four indices are present in Table 6.
We list 12 groups, a total of 2 hours of wind speed forecasting results in two methods, under the same phase space reconstruction. Compare the prediction performance in the next 10 min, 20 min, 30 min, and up to, 120 min.
Comparisons among several prediction results in two methods are present in Table 7.
Figures 1–5 show that relative error of wind speed prediction by EKF neural network is much smaller than that by BP neural network, through observing the future wind speed prediction of 150 groups. As can be seen in Table 6, the prediction effects are largely different by different kinds of phase space reconstruction methods. Four performance indices, which are Mean Absolute Error (MAE), Mean Relative Error (MRE), Mean Square Error (MSE), and Sum of Squared Error (SSE), of EKF neural network, are also far less than those of corresponding general neural network.
Apparently, EKF neural network can solve the inconsistency problem of phase space reconstruction and approximate chaotic time series well through subspace. The neural network model based on EKF has outstanding adaptability, so it can predict the wind speed chaotic time series with higher precision, compared with BP neural network.
Furthermore, we can conclude that in Table 7, prediction accuracy of EKF neural network is higher than that of BP neural network, by comparing the prediction performance of wind speed in the next 10 min, 20 min, 30 min, and up to, 120 min. It demonstrates that EKF neural network model, which has better dynamic adaptability, can better the prediction of wind speed time series with nonlinear chaotic characteristics. Therefore, the proposed phase space reconstruction method of the adaptive evolution approximation in this paper is an effective approach.
5. Conclusion and Further Work
The phase space reconstruction cannot meet characteristics of the completeness and independence, and the results with different reconstruction methods are obviously inconsistent. The reconstructed phase space is a subspace of the ideal space. If a subspace approximation can make the real-time dynamic evolution, then the initial constructed phase space, for which the evolution is adaptive subspace approximation, can finally approximate to the ideal phase space much better.
In this paper, neural network model based on nonlinear Kalman filter is established, by dynamic adaptivity of nonlinear Kalman filter. The model will add new samples in real time and gradually eliminate previous data, as a moving samples window, and the evolution of the training sample continually updates weights and thresholds of the neural network. As a result, adaptive subspace approximation is implemented by reconstructed incomplete phase space.
The optimized plan, which combines the nonlinear Kalman filter with neural network, sufficiently utilizes the nonlinear approximation capability of neural network and dynamic adaptive ability of real-time update correction of nonlinear Kalman filter. Consequently, it can realize subspace adaptive evolution approximation and solve the inconsistency problem of phase space reconstruction. Therefore, it is a nice direction in research into chaotic prediction. Future research can be performed in a number of areas. It provides a good technical support in studying problems of meteorology, hydrology, and finance fields.
This work was supported by the National Science Foundation of China (no. 51075418), the National Science Foundation of China (no. 61174015), Chongqing CMEC Foundations of China (no. CSTC 2013jjB40007), and Chongqing Scientific Personnel Training Plan of China (no. CSTC 2013kjrc-qnrc40008).
- G. Kaddoum, F. Gagnon, P. Chargé, and D. Roviras, “A generalized BER prediction method for differential chaos shift keying system through different communication channels,” Wireless Personal Communications, vol. 64, no. 2, pp. 425–437, 2012.
- B. Sivakumar, “Chaos theory in geophysics: past, present and future,” Chaos, Solitons & Fractals, vol. 19, no. 2, pp. 441–462, 2004.
- P. A. Mastorocostas, J. B. Theocharis, and A. G. Bakirtzis, “Fuzzy modeling for short term load forecasting using the orthogonal least squares method,” IEEE Transactions on Power Systems, vol. 14, no. 1, pp. 29–36, 1999.
- H. Y. Yang, H. Ye, G. Wang, J. Khan, and T. Hu, “Fuzzy neural very-short-term load forecasting based on chaotic dynamics reconstruction,” Chaos, Solitons & Fractals, vol. 29, no. 2, pp. 462–469, 2006.
- B. Sivakumar, “A phase-space reconstruction approach to prediction of suspended sediment concentration in rivers,” Journal of Hydrology, vol. 258, no. 1–4, pp. 149–162, 2002.
- A. Porporato and L. Ridolfi, “Nonlinear analysis of river flow time sequences,” Water Resources Research, vol. 33, no. 6, pp. 1353–1367, 1997.
- P. Zhao, L. Xing, and J. Yu, “Chaotic time series prediction: from one to another,” Physics Letters A, vol. 373, no. 25, pp. 2174–2177, 2009.
- H. D. I. Abarbanel, Analysis of Observed Chaotic Data, Springer, New York, NY, USA, 1996.
- M. B. Kennel, R. Brown, and H. D. I. Abarbanel, “Determining embedding dimension for phase-space reconstruction using a geometrical construction,” Physical Review A, vol. 45, no. 6, pp. 3403–3411, 1992.
- H. Ma and C. Han, “Selection of embedding dimension and delay time in phase space reconstruction,” Frontiers of Electrical and Electronic Engineering in China, vol. 1, no. 1, pp. 111–114, 2006.
- B. Chen, G. Liu, J. Tang et al., “Research on chaotic sequence autocorrelation by phase space method,” Journal of the University of Electronic Science and Technology of China, vol. 39, no. 6, pp. 859–863, 2010.
- A. Jiang, X. Huang, Z. Zhang, J. Li, Z.-Y. Zhang, and H.-X. Hua, “Mutual information algorithms,” Mechanical Systems and Signal Processing, vol. 24, no. 8, pp. 2947–2960, 2010.
- N. Abu-Shikhah and F. Elkarmi, “Medium-term electric load forecasting using singular value decomposition,” Energy, vol. 36, no. 7, pp. 4259–4271, 2011.
- C. Gao and X. Liu, “Chaotic identification of BF ironmaking process I: the calculation of saturated correlative dimension,” Acta Metallurgica Sinica, vol. 40, no. 4, pp. 347–350, 2004.
- I. M. Carrión and E. A. Antúnez, “Thread-based implementations of the false nearest neighbors method,” Parallel Computing, vol. 35, no. 10-11, pp. 523–534, 2009.
- J. Hite Jr., Learning in Chaos, Gulf Professional Publishing, Amsterdam, The Netherlands, 1999.
- Y. Hu and T. Chen, “Phase-space reconstruction technology of chaotic attractor based on C-C method,” Journal of Electronic Measurement and Instrument, vol. 35, no. 10-11, pp. 425–430, 2012.
- D. A. Fadare, “The application of artificial neural networks to mapping of wind speed profile for energy application in Nigeria,” Applied Energy, vol. 87, no. 3, pp. 934–942, 2010.
- S. Salcedo-Sanz, A. M. Ángel M. Pérez-Bellido, E. G. Ortiz-García, A. Portilla-Figueras, L. Prieto, and D. Paredes, “Hybridizing the fifth generation mesoscale model with artificial neural networks for short-term wind speed prediction,” Renewable Energy, vol. 34, no. 6, pp. 1451–1457, 2009.
- D. C. Dracopoulos, Evolutionary Learning Algorithms for Neural Adaptative Control, Springer, London, UK, 1997.
- J. Xue and Z. Shi, “Short-time traffic flow prediction based on chaos time series theory,” Journal of Transportation Systems Engineering and Information Technology, vol. 8, no. 5, pp. 68–72, 2008.
- H. Yang, J. Li, and F. Ding, “A neural network learning algorithm of chemical process modeling based on the extended Kalman filter,” Neurocomputing, vol. 70, no. 4–6, pp. 625–632, 2007.
- R. S. Bucy and K. D. Senne, “Digital synthesis of non-linear filters,” Automatica, vol. 7, no. 3, pp. 287–298, 1971.
- J. Wang and Y. Xie, “Solar radiation prediction based on phase space reconstruction of wavelet neural network,” Procedia Engineering, vol. 15, no. 1, pp. 4603–4607, 2011.
- H. Kantz and T. Schreiber, Nonlinear Time Series Analysis, Cambridge University Press, Cambridge, UK, 1997.
- T. E. Karakasidis and A. Charakopoulos, “Detection of low-dimensional chaos in wind time series,” Chaos, Solitons & Fractals, vol. 41, no. 4, pp. 1723–1732, 2009.
- P. Louka, G. Galanis, N. Siebert et al., “Improvements in wind speed forecasts for wind power prediction purposes using Kalman filtering,” Journal of Wind Engineering and Industrial Aerodynamics, vol. 96, no. 12, pp. 2348–2362, 2008.
- M. Poncela, P. Poncela, and J. R. Perán, “Automatic tuning of Kalman filters by maximum likelihood methods for wind energy forecasting,” Applied Energy, vol. 108, no. 12, pp. 349–362, 2013.