Journal of Applied Mathematics

Journal of Applied Mathematics / 2013 / Article
Special Issue

Mathematical Approaches in Advanced Control Theories 2013

View this Special Issue

Research Article | Open Access

Volume 2013 |Article ID 983051 | https://doi.org/10.1155/2013/983051

Tai-fu Li, Wei Jia, Wei Zhou, Ji-ke Ge, Yu-cheng Liu, Li-zhong Yao, "Incomplete Phase Space Reconstruction Method Based on Subspace Adaptive Evolution Approximation", Journal of Applied Mathematics, vol. 2013, Article ID 983051, 9 pages, 2013. https://doi.org/10.1155/2013/983051

Incomplete Phase Space Reconstruction Method Based on Subspace Adaptive Evolution Approximation

Academic Editor: Baocang Ding
Received19 Jul 2013
Accepted19 Sep 2013
Published10 Oct 2013

Abstract

The chaotic time series can be expanded to the multidimensional space by phase space reconstruction, in order to reconstruct the dynamic characteristics of the original system. It is difficult to obtain complete phase space for chaotic time series, as a result of the inconsistency of phase space reconstruction. This paper presents an idea of subspace approximation. The chaotic time series prediction based on the phase space reconstruction can be considered as the subspace approximation problem in different neighborhood at different time. The common static neural network approximation is suitable for a trained neighborhood, but it cannot ensure its generalization performance in other untrained neighborhood. The subspace approximation of neural network based on the nonlinear extended Kalman filtering (EKF) is a dynamic evolution approximation from one neighborhood to another. Therefore, in view of incomplete phase space, due to the chaos phase space reconstruction, we put forward subspace adaptive evolution approximation method based on nonlinear Kalman filtering. This method is verified by multiple sets of wind speed prediction experiments in Wulong city, and the results demonstrate that it possesses higher chaotic prediction accuracy.

1. Introduction

In recent years, industrial disasters and accidents occurred frequently, the meteorological and hydrological conditions were complicated and changeable, and financial markets fluctuated drastically. These phenomena often contain chaotic characteristics [1, 2], and prediction [3] for these phenomena is imminent. For a long time, there was no scientific tool for handling this issue, because the changing mechanisms of characteristics in these phenomena were not understood very well. Hence, aiming at the chaotic characteristics, some scholars worked with structures and made a lot of new researches on the prediction of chaotic time series [48].

To study and deal with the measurement data of chaotic system, Kennel et al. presented the reconstruction method of phase space system. Two parameters, the embedding dimension and delay time , needed to be determined before the phase space reconstruction [9, 10]. At present, time delay selection methods that are commonly used in the chaotic short-term prediction mainly include autocorrelation method [11], mutual information method [12], and singular value fraction method [13]. Calculating methods of embedding dimension mainly include saturated correlation dimension [14], false nearest neighbors method [15], and Cao's method [16]. Hu and Chen put forward the C-C method [17], which can simultaneously estimate the delay time and embedding dimension with the correlation integral. Autocorrelation method extracts only linear correlation degree between time series, which is hard to be applied to high-dimensional chaos system and nonlinear dynamical system. Mutual information method, which can determine the optimal delay time by calculating the first minimal value of mutual information function, is a nonlinear analysis method, but it cannot avoid massive calculation and cannot satisfy the requirement of complicated space division. It is difficult to determine the threshold of the singular value, because the singular value fraction method is largely affected by noise. When the embedded dimension with the saturated correlation dimension is calculated, the main question is to choose the different neighborhood radius. The radius selection has certain randomness, and the result will be in large deviation with improper choice, because of the influence of noise in the data and excessive concentration of the data. The determination of threshold has very strong subjectivity when we use false nearest neighbors method to determine the embedding dimension. There is no objective standard to determine the threshold value, especially for the experimental data, which may get a wrong result. Cao's method, an improved false nearest neighbors method, can effectively distinguish random signals and deterministic signals, and embedding dimension can be obtained through a less amount of data. C-C method is based on the statistical theory, so cannot be precisely determined.

Researches have showed that different phase space reconstruction methods get different and . Moreover, the same chaotic time series with the same kind of method in different times may get different and . There is no phase space reconstruction that can obtain complete and independent phase space. After phase space reconstruction, prediction model is often established through the functional approximation method.

The prediction model based on phase space reconstruction has been used to adopt the functional approximation method based on the neural network [1821], which has strong nonlinear fitting capability and can approximate any complex nonlinear relationships. However, since neural network is only suitable for approximation of a deterministic system, it is difficult to guarantee the time-varying system performance and ensure its generalization performance in other untrained neighborhood. Meanwhile, the prediction effect of neural network is not good, because the chaotic time series is a complex nonlinear uncertain system.

In this study, we introduce Kalman filtering to neural network model [22], inspired by Kalman iteration and Bucy and Sunahara’s nonlinear extended Kalman filtering theory [23]. The subspace approximation of neural network based on the nonlinear extended Kalman filtering (EKF) has a function which is dynamic evolution approximation from one neighborhood to another. Therefore, we can constitute a phase space by choosing a kind of phase space reconstruction method, and the space may be incomplete, not separate, and can be seen as a subspace of the ideal phase space. On this basis, we put forward adaptive neural network model based on nonlinear Kalman filtering and finally realize the subspace approximation of dynamic evolution system. In addition, we simulate wind speed series in Wulong city using the proposed method. By comparing with BP neural network prediction model, the results show that our method possesses higher prediction accuracy.

The paper is organized as follows. Section 2 discusses about the subspace approximation of phase space reconstruction. In Section 3, we describe the neural network model based on nonlinear Kalman filtering. Section 4 uses practical examples and series tests to verify the proposed method, while Section 5 contains the conclusions of the present work.

2. Subspace Approximation of Phase Space Reconstruction

Reconstructing phase space by chaos theory needs to identify the chaos of time series. Single variable time series can be reconstructed into a phase space by Takens’ embedding theorem in phase space reconstruction [24, 25]; that is, the original dynamical system can be restored in the sense of topological equivalence as long as the embedding dimension is sufficiently high. For the observed time series , after time delay reconstruction by Takens embedding theorem, it will receive a set of space vector After phase space reconstruction, the data space is Accordingly, we acquire where is a single-valued function. Then, we have However, it cannot be really obtained as the data are often limited. Hence, can only be constituted by limited measurement data, making sufficiently approximate to , consequently we can get a nonlinear prediction model.

This paper employs the neural network to predict chaotic series. However, the neural network cannot readily handle the inconsistency of the phase space reconstruction because of uncertain nonlinear chaotic time series. Therefore, it is crucial to adaptively construct subspace to approximate chaotic series through the incomplete phase space. The feature of adaptive subspace approximation is that it can add new data in real time and forget old data in the process of training. Consequently, weights and thresholds of the neural network are continuously modified to realize the dynamic evolution modeling.

3. Neural Network Model Based on Nonlinear Kalman Filtering

Kalman filtering has good adaptability. It can dynamically update and forecast the system information in real time with limited data. However, it cannot be readily used for complicated nonlinear model. Meanwhile, the extended Kalman filtering (EKF) is a kind of effective method to handle nonlinear filtering.

The mathematical model of EKF is as follows: where and are independent, zero mean, and Gaussian random processes with covariance matrices and , respectively. The statistical properties are as follows:

EKF spreads nonlinear functions and to Taylor series around filtering value and predicted value , respectively, only retaining the first-order information. Hence, the linearization model of the nonlinear system is obtained, and then we can obtain the EKF formula in nonlinear system by basic equations of Kalman filtering.

Given a forward network with layers, the numbers of neurons in each layer are   . Suppose that input layer is the first layer and output layer is the layer. The weights of the layer neurons are   . In order to convert the calculation of connection weights in the above problem into filter recursive estimation form, we let all of the network weights constitute the state vector where state vector consists of all of the weights according to the linear array, and its dimension is as follows: Then the state equation and measurement equation of the system can be expressed as where is the expected output, is the input vector, and is the actual output.

The measurement noise is assumed to be additive, white, and Gaussian, with zero mean and with covariance matrix defined by Suppose that the output of the node for the layer in the iteration is From (10) and (12), we have Assume that Accordingly, the measurement equation may also be expressed as The Jacobian matrix of the function is described by Similarly, all thresholds of the network constitute the state vector where the dimension is Suppose that and are both state variable; that is, the state vector composed of weights and thresholds is described by Kalman filtering algorithm on training weights and thresholds of the neural network is as in Table 1.


Extended Kalman filtering neural network

(1) Initialization
(2) Time update (forecast)
(3) Measurement update (correct)


and are process noise covariance and measurement noise covariance, respectively, is the Jacobian matrix of observable model, is the optimal predictive value for step according to step , and is the optimal filter estimate for step .

4. Simulation Examples

4.1. Determining of Embedding Dimension and Delay Time

One of the most popular chaos logistic mapper is selected as the study object. Logistic equation is

The related time series are produced according to (20). It is a chaotic system when . Assume that initial value of series is 0.1, and 4000 points are calculated. The first 1000 points are eliminated as transition phenomenon, leaving the remaining 3000 points to reconstruct phase space. Before the phase space reconstruction, we determine the embedding dimension and delay time . A comparison among several methods is present in Table 2.


ParameterMethod
AutocorrelationMutual informationFalse nearest neighborsCaoC-C

1125
435

“—” means nothing.

Obviously, the optimal embedding dimension and delay time are generally different by different methods of phase space reconstruction.

In order to verify the fact that data at different time will obtain different embedding dimension and delay time with the same phase space reconstruction method, we have the following experiment.

The remaining 3000 points () are divided into five parts, with time intervals , , , , and , respectively. Embedding dimension and delay time are present in Table 3 by C-C method.


ParameterInterval






53445
554910

Apparently, the data during different time periods will acquire different embedding dimension and delay time by using the same phase space reconstruction method.

4.2. Wind Speed Chaotic Series Forecasting Simulation

Analysis about the chaotic characteristics of wind speed in the process of wind power generation has been presented in a related article [26]. We record one of the wind speed data every 10 minutes, and 150 groups of wind speed data in Wulong city are used to simulate experiments in our study. We obtain the corresponding and by different phase space reconstruction methods, as shown in Table 4.


ParameterMethod
AutocorrelationMutual informationFalse nearest neighborsCaoC-C

1123
m 374

Various combinations are present in Table 5.


ModelCombinationParameterForecasting

a1Autocorrelation + false nearest neighbors , BPNN
b1Autocorrelation + false nearest neighbors , EKFNN
a2Mutual information + false nearest neighbors , BPNN
b2Mutual information + false nearest neighbors , EKFNN
a3Autocorrelation + Cao , BPNN
b3Autocorrelation + Cao , EKFNN
a4Mutual information + Cao , BPNN
b4Mutual information + Cao , EKFNN
a5C-C , BPNN
b5C-C , EKFNN

Wind speed prediction [27, 28] of chaotic time series about neural network model usually extracts phase space reference points as the BP neural network training samples on the basis of phase space reconstruction. We establish the neural network model based on nonlinear Kalman filtering, including two parts: predict wind speed and constantly modify weights and thresholds of the neural network by Kalman recursion. In this paper, BPNN model in the same structure is employed to forecast wind speed time series, in order to illustrate the validity of EKFNN on predicting the chaotic time series. The same 150 groups of wind speed data are used to simulate experiments. The predicted curves and error curves are shown in Figures 1, 2, 3, 4, and 5.

Comparisons among several models in four indices are present in Table 6.


ModelError
MAEMREMSESSE

a10.84820.10761.3786206.7967
b10.33650.04160.191928.7834
a21.22790.16742.9614444.2152
b20.61900.07590.620993.1357
a30.51060.06560.408961.3361
b30.37830.04580.265439.8130
a42.40050.30798.4766
b40.08520.01100.01772.6584
a50.89520.11731.7970269.5495
b50.28670.03570.135920.3850

MAE, MRE, MSE, and SSE are Mean Absolute Error, Mean Relative Error, Mean Square Error, and Sum of Squared Error, respectively.

We list 12 groups, a total of 2 hours of wind speed forecasting results in two methods, under the same phase space reconstruction. Compare the prediction performance in the next 10 min, 20 min, 30 min, and up to, 120 min.

Comparisons among several prediction results in two methods are present in Table 7.


Time (min)Observed value (m/s)Predicted value (m/s)Relative error (%)Predicted value (m/s)Relative error (%)

a1b1

1010.490010.08013.956110.51010.1823
2010.37009.97833.776210.40110.2630
3010.690010.10245.490210.46212.1476
4010.210010.88026.537410.62024.0235
5010.30009.142411.240210.15311.4947
609.800010.35085.640110.22104.2446
709.30008.68816.58109.60323.2585
809.49008.188313.72019.31631.8295
9010.11009.60135.03219.76313.4282
1009.210010.472613.720010.03008.9132
1108.55007.208215.69038.86213.6533
1209.00007.082421.31018.54705.0347

a2b2

1010.49009.69307.597710.47850.1095
2010.370010.40060.295510.42490.5292
3010.690010.04895.996810.49921.7845
4010.210010.56993.525010.33571.2311
5010.300010.25590.428510.25180.4675
609.800010.14453.515110.11323.1960
709.30009.25830.448810.07038.2826
809.49009.89444.26099.95474.8963
9010.110010.69375.77399.75033.5580
1009.210011.442024.23519.52123.3787
1108.550011.831538.38049.18237.3954
1209.000011.254325.04759.30123.3463

a3b3

1010.490010.97754.646810.71942.1870
2010.370010.49311.187210.42770.5567
3010.690010.38782.826910.41682.5559
4010.210013.739334.567210.18930.2032
5010.30009.80854.771610.29180.0793
609.800012.182624.312610.09933.0540
709.30009.79245.29439.83705.7739
809.49009.35211.45299.66891.8855
9010.110014.580344.21689.77723.2920
1009.210014.609558.62649.58054.0233
1108.55008.64250.08219.34429.2884
1209.00009.73798.19889.33143.6817

a4b4

1010.49008.929514.875910.11373.5868
2010.37009.203711.24669.73446.1296
3010.69008.649419.088710.33083.3597
4010.21008.551616.24269.98032.2501
5010.30009.30389.671710.56852.6064
609.80008.746710.747910.11833.2477
709.30008.71266.31619.77795.1391
809.49008.61889.18029.84133.7021
9010.11009.47136.317610.98868.6901
1009.21009.32041.19928.88163.5657
1108.55009.502911.14528.24973.5118
1209.00009.69097.67688.85581.6017

a5b5

1010.490010.12273.501710.42270.6412
2010.370010.51501.398510.48901.1473
3010.690010.37142.980510.62680.5909
4010.210010.16250.465210.56383.4649
5010.300010.52362.170510.47781.7260
609.800010.53667.515910.17413.8174
709.30009.87486.18079.82245.6174
809.49009.45000.42179.67441.9432
9010.11009.72793.77949.89502.1267
1009.21008.97282.57559.61954.4458
1108.55009.38389.75199.24648.1455
1209.00009.33283.69739.07100.7889

Figures 15 show that relative error of wind speed prediction by EKF neural network is much smaller than that by BP neural network, through observing the future wind speed prediction of 150 groups. As can be seen in Table 6, the prediction effects are largely different by different kinds of phase space reconstruction methods. Four performance indices, which are Mean Absolute Error (MAE), Mean Relative Error (MRE), Mean Square Error (MSE), and Sum of Squared Error (SSE), of EKF neural network, are also far less than those of corresponding general neural network.

Apparently, EKF neural network can solve the inconsistency problem of phase space reconstruction and approximate chaotic time series well through subspace. The neural network model based on EKF has outstanding adaptability, so it can predict the wind speed chaotic time series with higher precision, compared with BP neural network.

Furthermore, we can conclude that in Table 7, prediction accuracy of EKF neural network is higher than that of BP neural network, by comparing the prediction performance of wind speed in the next 10 min, 20 min, 30 min, and up to, 120 min. It demonstrates that EKF neural network model, which has better dynamic adaptability, can better the prediction of wind speed time series with nonlinear chaotic characteristics. Therefore, the proposed phase space reconstruction method of the adaptive evolution approximation in this paper is an effective approach.

5. Conclusion and Further Work

The phase space reconstruction cannot meet characteristics of the completeness and independence, and the results with different reconstruction methods are obviously inconsistent. The reconstructed phase space is a subspace of the ideal space. If a subspace approximation can make the real-time dynamic evolution, then the initial constructed phase space, for which the evolution is adaptive subspace approximation, can finally approximate to the ideal phase space much better.

In this paper, neural network model based on nonlinear Kalman filter is established, by dynamic adaptivity of nonlinear Kalman filter. The model will add new samples in real time and gradually eliminate previous data, as a moving samples window, and the evolution of the training sample continually updates weights and thresholds of the neural network. As a result, adaptive subspace approximation is implemented by reconstructed incomplete phase space.

The optimized plan, which combines the nonlinear Kalman filter with neural network, sufficiently utilizes the nonlinear approximation capability of neural network and dynamic adaptive ability of real-time update correction of nonlinear Kalman filter. Consequently, it can realize subspace adaptive evolution approximation and solve the inconsistency problem of phase space reconstruction. Therefore, it is a nice direction in research into chaotic prediction. Future research can be performed in a number of areas. It provides a good technical support in studying problems of meteorology, hydrology, and finance fields.

Acknowledgments

This work was supported by the National Science Foundation of China (no. 51075418), the National Science Foundation of China (no. 61174015), Chongqing CMEC Foundations of China (no. CSTC 2013jjB40007), and Chongqing Scientific Personnel Training Plan of China (no. CSTC 2013kjrc-qnrc40008).

References

  1. G. Kaddoum, F. Gagnon, P. Chargé, and D. Roviras, “A generalized BER prediction method for differential chaos shift keying system through different communication channels,” Wireless Personal Communications, vol. 64, no. 2, pp. 425–437, 2012. View at: Publisher Site | Google Scholar
  2. B. Sivakumar, “Chaos theory in geophysics: past, present and future,” Chaos, Solitons & Fractals, vol. 19, no. 2, pp. 441–462, 2004. View at: Publisher Site | Google Scholar | MathSciNet
  3. P. A. Mastorocostas, J. B. Theocharis, and A. G. Bakirtzis, “Fuzzy modeling for short term load forecasting using the orthogonal least squares method,” IEEE Transactions on Power Systems, vol. 14, no. 1, pp. 29–36, 1999. View at: Publisher Site | Google Scholar
  4. H. Y. Yang, H. Ye, G. Wang, J. Khan, and T. Hu, “Fuzzy neural very-short-term load forecasting based on chaotic dynamics reconstruction,” Chaos, Solitons & Fractals, vol. 29, no. 2, pp. 462–469, 2006. View at: Publisher Site | Google Scholar
  5. B. Sivakumar, “A phase-space reconstruction approach to prediction of suspended sediment concentration in rivers,” Journal of Hydrology, vol. 258, no. 1–4, pp. 149–162, 2002. View at: Publisher Site | Google Scholar
  6. A. Porporato and L. Ridolfi, “Nonlinear analysis of river flow time sequences,” Water Resources Research, vol. 33, no. 6, pp. 1353–1367, 1997. View at: Google Scholar
  7. P. Zhao, L. Xing, and J. Yu, “Chaotic time series prediction: from one to another,” Physics Letters A, vol. 373, no. 25, pp. 2174–2177, 2009. View at: Publisher Site | Google Scholar
  8. H. D. I. Abarbanel, Analysis of Observed Chaotic Data, Springer, New York, NY, USA, 1996. View at: Publisher Site | MathSciNet
  9. M. B. Kennel, R. Brown, and H. D. I. Abarbanel, “Determining embedding dimension for phase-space reconstruction using a geometrical construction,” Physical Review A, vol. 45, no. 6, pp. 3403–3411, 1992. View at: Publisher Site | Google Scholar
  10. H. Ma and C. Han, “Selection of embedding dimension and delay time in phase space reconstruction,” Frontiers of Electrical and Electronic Engineering in China, vol. 1, no. 1, pp. 111–114, 2006. View at: Publisher Site | Google Scholar
  11. B. Chen, G. Liu, J. Tang et al., “Research on chaotic sequence autocorrelation by phase space method,” Journal of the University of Electronic Science and Technology of China, vol. 39, no. 6, pp. 859–863, 2010. View at: Publisher Site | Google Scholar
  12. A. Jiang, X. Huang, Z. Zhang, J. Li, Z.-Y. Zhang, and H.-X. Hua, “Mutual information algorithms,” Mechanical Systems and Signal Processing, vol. 24, no. 8, pp. 2947–2960, 2010. View at: Publisher Site | Google Scholar
  13. N. Abu-Shikhah and F. Elkarmi, “Medium-term electric load forecasting using singular value decomposition,” Energy, vol. 36, no. 7, pp. 4259–4271, 2011. View at: Publisher Site | Google Scholar
  14. C. Gao and X. Liu, “Chaotic identification of BF ironmaking process I: the calculation of saturated correlative dimension,” Acta Metallurgica Sinica, vol. 40, no. 4, pp. 347–350, 2004. View at: Google Scholar
  15. I. M. Carrión and E. A. Antúnez, “Thread-based implementations of the false nearest neighbors method,” Parallel Computing, vol. 35, no. 10-11, pp. 523–534, 2009. View at: Publisher Site | Google Scholar
  16. J. Hite Jr., Learning in Chaos, Gulf Professional Publishing, Amsterdam, The Netherlands, 1999.
  17. Y. Hu and T. Chen, “Phase-space reconstruction technology of chaotic attractor based on C-C method,” Journal of Electronic Measurement and Instrument, vol. 35, no. 10-11, pp. 425–430, 2012. View at: Google Scholar
  18. D. A. Fadare, “The application of artificial neural networks to mapping of wind speed profile for energy application in Nigeria,” Applied Energy, vol. 87, no. 3, pp. 934–942, 2010. View at: Publisher Site | Google Scholar
  19. S. Salcedo-Sanz, A. M. Ángel M. Pérez-Bellido, E. G. Ortiz-García, A. Portilla-Figueras, L. Prieto, and D. Paredes, “Hybridizing the fifth generation mesoscale model with artificial neural networks for short-term wind speed prediction,” Renewable Energy, vol. 34, no. 6, pp. 1451–1457, 2009. View at: Publisher Site | Google Scholar
  20. D. C. Dracopoulos, Evolutionary Learning Algorithms for Neural Adaptative Control, Springer, London, UK, 1997.
  21. J. Xue and Z. Shi, “Short-time traffic flow prediction based on chaos time series theory,” Journal of Transportation Systems Engineering and Information Technology, vol. 8, no. 5, pp. 68–72, 2008. View at: Google Scholar
  22. H. Yang, J. Li, and F. Ding, “A neural network learning algorithm of chemical process modeling based on the extended Kalman filter,” Neurocomputing, vol. 70, no. 4–6, pp. 625–632, 2007. View at: Publisher Site | Google Scholar
  23. R. S. Bucy and K. D. Senne, “Digital synthesis of non-linear filters,” Automatica, vol. 7, no. 3, pp. 287–298, 1971. View at: Google Scholar
  24. J. Wang and Y. Xie, “Solar radiation prediction based on phase space reconstruction of wavelet neural network,” Procedia Engineering, vol. 15, no. 1, pp. 4603–4607, 2011. View at: Publisher Site | Google Scholar
  25. H. Kantz and T. Schreiber, Nonlinear Time Series Analysis, Cambridge University Press, Cambridge, UK, 1997. View at: MathSciNet
  26. T. E. Karakasidis and A. Charakopoulos, “Detection of low-dimensional chaos in wind time series,” Chaos, Solitons & Fractals, vol. 41, no. 4, pp. 1723–1732, 2009. View at: Publisher Site | Google Scholar
  27. P. Louka, G. Galanis, N. Siebert et al., “Improvements in wind speed forecasts for wind power prediction purposes using Kalman filtering,” Journal of Wind Engineering and Industrial Aerodynamics, vol. 96, no. 12, pp. 2348–2362, 2008. View at: Publisher Site | Google Scholar
  28. M. Poncela, P. Poncela, and J. R. Perán, “Automatic tuning of Kalman filters by maximum likelihood methods for wind energy forecasting,” Applied Energy, vol. 108, no. 12, pp. 349–362, 2013. View at: Publisher Site | Google Scholar

Copyright © 2013 Tai-fu Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

1034 Views | 682 Downloads | 1 Citation
 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.