Nonlinear Time Series: Computations and ApplicationsView this Special Issue
On the Predictability of Long-Range Dependent Series
This paper points out that the predictability analysis of conventional time series may in general be invalid for long-range dependent (LRD) series since the conventional mean-square error (MSE) may generally not exist for predicting LRD series. To make the MSE of LRD series prediction exist, we introduce a generalized MSE. With that, the proof of the predictability of LRD series is presented in Hilbert space.
Let be a realization, which is a second-order random function for . Let be a given sample of for . Then, one of the important problems in time series is to predict or forecast for based on the known realizations of ; see, for example, Clements and Hendry , Box et al. , and Fuller .
A well-known case in the field of time series prediction refers to Yule’s work for the analysis of Wolfer's sunspot numbers (Yule ). The early basic theory of predicting a 2nd-order stationary random function in the conventional sense refers to the work of Wiener  and Kolmogorov . By conventional sense, we mean that the stationary random functions Wiener and Kolmogorov considered are not long-range dependent (LRD). In other words, the time series they studied have finite mean and variance. Consequently, they in general are not heavy tailed as can be seen from Zadeh and Ragazzini , Bhansali , and Robinson .
The predictability of conventional time series has been well studied; see, for example, Papoulis , Vaidyanathan , Bhansali , Lyman et al. , Lyman and Edmonson , and Dokuchaev . The basic idea in this regard is to use mean-square error (MSE) as a constraint to obtain a prediction; see, for example, Harrison et al. , Bellegem and Sachs , Man , as well as Clements and Hendry . We shall note in next section that the conventional MSE may in general fail to be used for predicting LRD series.
LRD processes gain increasing applications in various fields of sciences and technologies; see, for example, Beran , Mandelbrot , Cattani et al. , and Li et al. [23–29]. Consequently, the prediction is desired for LRD series. The literature regarding the prediction of LRD series appears to be increasing; see, for example, Brodsky and Hurvich , Reisen and Lopes , Bisaglia and Bordignon , Bhansali and Kokoszka , Man , Bayraktar et al. , Man and Tiao , Bisaglia and Gerolimetto , Godet , as well as Gooijer and Hyndman . However, unfortunately, suitable MSE used for predicting LRD series may be overlooked, leaving a pitfall in this respect. We shall present a generalized MSE in the domain of generalized functions for the purpose of proving the existence of LRD series prediction.
The rest of this article is arranged as follows. Section 2 will point out the pitfall of prediction of time series based on traditional MSE. The proof of the predictability of LRD series will be proposed in Section 3, which is followed by discussions and conclusions.
2. Problem Statement
Denote the autocorrelation function (ACF) of by , where . Then, is called short-range dependent (SRD) series if is integrable (Beran ), that is,
On the other side, is long-range dependent (LRD) if is nonintegrable, that is,
A typical form of such an ACF has the following asymptotic expression:
where is a constant and .
Denote the probability density function (PDF) of by . Then, the ACF of can be expressed by
Considering that is nonintegrable, we see that a heavy-tailed PDF is a consequence of LRD series; see, for example, Resnick , Heath et al. , Paxson and Floyd , Li [23, 24, 43], Abry et al. , as well as Adler et al. .
Denote by the mean of . Then,
The variance of is given by
One remarkable thing in LRD series is that the tail of p(x) may be so heavy that the above integral either (2.5) or (2.6) does not exist (Bassingthwaighte et al. , Doukhan et al. , Li ). To explain this, we utilize the Pareto distribution. Denote the PDF of the Pareto distribution. Then (G. A. Korn and T. M. Korn ),
where . The mean and variance of that follows are, respectively, given by
It can be easily seen that and do not exist if .
Following the work of Kolmogorov’s, a linear prediction can be expressed as follows. Given and , the selection of proper real coefficient as is such that the following linear combination of random variables given by
By minimizing (2.11), one has the desired in (2.10). Wiener well studied that criterion for both prediction and filtering; see, for example, Levinson [50, 51]. A predictor following (2.10) and (2.11) can be regarded in the class of Wiener-Kolmogorov predictors.
Various forms of linear combination in terms of (2.10) have been developed, such as autoregressive moving average (ARMA) model, autoregressive (AR) model, moving average (MA) model, and autoregressive integrated moving average (ARIMA); see, for example, Lyman et al. , Lyman and Edmonson , Wolff et al. , Bhansali [12, 53], Markhoul , Kohn and Ansley , Zimmerman and Cressie , Peiris and Perera , Kudritskii , Bisaglia and Bordignon , Kim , Cai , Harvill and Ray , Atal , Huang , Schick and Wefelmeyer , Jamalizadeh and Balakrishnan , Clements and Hendry , and Box et al. . However, one thing in common for different forms of predictors is to minimize prediction error that in principle usually follows the form of (2.11).
Note that the necessary condition for the above-described Wiener-Kolmogorov predictor to be valid is that exists (Kolmogorov ). For LRD series, however, it may not always be satisfied. For instance, if an LRD series obeys the Pareto distribution, its mean does not exist for ; see (2.8).
In addition to the fact that the mean of an LRD series may not exist, its variance may not exist either. The error in (2.11) can be expressed by
Kolmogorov stated that the above does not increase as increases . However, that statement may be untrue if is LRD.
It is worth noting that errors may be heavy tailed; see, for example, Peng and Yao  as well as Hall and Yao . For instance, LRD teletraffic is heavy tailed with the possible heavy-tail model of Pareto (Resnick , Michiel and Laevens ) and it is Gaussian at large time scales (Paxson and Floyd , Scherrer et al. ). Therefore, it is quite reasonable to assume that follows a heavy-tailed distribution, for example, the Pareto distribution, for the purpose of this presentation. If it obeys the Pareto one, then the above expression approaches infinite for (see (2.9)) no matter how large is.
From the above discussions, we see that it may be unsuitable to use the conventional MSE as used in the class of conventional Wiener-Kolmogorov predictors to infer that LRD series is predictable. In the next section, we shall give the proof of the predictability of LRD series.
3. Predictability of LRD Series
Let , where is the set of LRD processes. Let Then, We now consider the norms and inner products in and .
Definition 3.1 (see ). A function of rapid decay is a smooth function such that as for all , where is the space of complex numbers. The set of all functions of rapid decay is denoted by .
In the discrete case, the rapid decayed function is denoted by and we still use the symbol S to specify the space it belongs to for the simplicity without confusions.
Lemma 3.2 (see ). Every function belonging to is absolutely integrable in the continuous case or absolutely summable in the discrete case.
Now, define the norm of by
where . Define the inner product of by
Then, combining any with its limit makes a Hilbert space.
Then, is the closed subset of .
Lemma 3.3 (see [73–75], Existence of a unique minimizing element in Hilbert space). Let be a Hilbert space and let be a closed convex subset of . Let , . Then, there exists a unique element satisfying
Theorem 3.4. Let L be a linear combination of the past values of according to (2.10). Then, there exists a unique such that
Proof. is a Hilbert space. is its closed subset and it is obviously convex. According to Lemma 3.3, for any there exists a unique such that
The above theorem exhibits that LRD series are predictable in the sense that the mean-square error expressed by (2.12) is in general generalized to the following for :
4. Discussions and Conclusions
LRD series considerably differ from the conventional series; see, for example, Beran [20, 76], Adler et al. , Doukhan et al. , as well as Künsch et al. . Examples mentioned in this regard are regressions for fitting LRD models (Peng and Yao , Beran , and Beran et al. ), variance analysis of autocorrelation estimation (Li and Zhao ), stationarity test (Li et al. ), power spectra (Li and Lim [82, 83]), and [84–91]. This paper addresses the particularity of the predictability of LRD series. We have given a proof of LRD series being predictable. As a side product obtained from the proof procedure, the mean-square error used by Kolmogorov as a criterion of LRD series prediction has been generalized to be the form of (3.5).
This work was partly supported by the National Natural Science Foundation of China (NSFC) under the project Grant numbers 60573125, 60873264, 60703112, and 60873168 and the National High Technology Research and Development Program (863) of China under Grant no. 2009AA01Z418.
G. E. P. Box, G. M. Jenkins, and G. C. Reinsel, Time Series Analysis: Forecasting and Control, Prentice-Hall, Englewood Cliffs, NJ, USA, 3rd edition, 1994.View at: MathSciNet
W. A. Fuller, Introduction to Statistical Time Series, Wiley Series in Probability and Statistics: Probability and Statistics, John Wiley & Sons, New York, NY, USA, 2nd edition, 1995.View at: MathSciNet
G. Udny Yule, “On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers,” Philosophical Transactions of the Royal Society A, vol. 226, no. 636–646, pp. 267–297, 1927.View at: Google Scholar
N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, John Wiley & Sons, New York, NY, USA; The MIT Press, Cambridge, Mass, USA, 1964.
A. N. Kolmogorov, “Interpolation and extrapolation of stationary random sequences,” Izvestiya Akademii Nauk SSSR, vol. 5, pp. 3–14, 1941.View at: Google Scholar
E. A. Robinson, “A historical perspective of spectrum estimation,” Proceedings of the IEEE, vol. 70, no. 9, pp. 885–907, 1982.View at: Google Scholar
A. Papoulis, “A note on the predictability of band-limited processes,” Proceedings of the IEEE, vol. 73, no. 8, pp. 1332–1333, 1985.View at: Google Scholar
P. P. Vaidyanathan, “On predicting a band-limited signal based on past sample values,” Proceedings of the IEEE, vol. 75, no. 8, pp. 1125–1159, 1987.View at: Google Scholar
R. J. Lyman, W. W. Edmonson, S. McCullough, and M. Rao, “The predictability of continuous-time, bandlimited processes,” IEEE Transactions on Signal Processing, vol. 48, no. 2, pp. 311–316, 2000.View at: Google Scholar
M. P. Clements and D. F. Hendry, “Forecasting economic processes,” International Journal of Forecasting, vol. 14, no. 1, pp. 111–131, 1998.View at: Google Scholar
J. Beran, Statistics for Long-Memory Processes, vol. 61 of Monographs on Statistics and Applied Probability, Chapman and Hall, New York, NY, USA, 1994.View at: MathSciNet
B. B. Mandelbrot, Gaussian Self-Affinity and Fractals, Springer, New York, NY, USA, 2001.View at: MathSciNet
M. Li, “Generation of teletraffic of generalized Cauchy type,” Physica Scripta, vol. 81, no. 2, p. 10, 2010.View at: Google Scholar
M. Li and W. Zhao, “Representation of a stochastic traffic bound,” to appear in IEEE Transactions on Parallel and Distributed Systems.View at: Google Scholar
J. Brodsky and C. M. Hurvich, “Multi-step forecasting for long-memory processes,” Journal of Forecasting, vol. 18, no. 1, pp. 59–75, 1999.View at: Google Scholar
R. J. Bhansali and P. S. Kokoszka, “Prediction of long-memory time series: a tutorial review,” in Processes with Long-Range Correlations, vol. 621 of Lecture Notes in Physics, pp. 3–21, Springer, Berlin, Germany, 2003.View at: Google Scholar
S. I. Resnick, Heavy-Tail Phenomena: Probabilistic and Statistical Modeling, Springer Series in Operations Research and Financial Engineering, Springer, New York, NY, USA, 2007.View at: MathSciNet
M. Li, “Fractional Gaussian noise and network traffic modeling,” in Proceedings of the 8th WSEAS International Conference on Applied Computer and Applied Computational Science, pp. 34–39, Hangzhou, China, May 2009.View at: Google Scholar
R. J. Adler, R. E. Feldman, and M. S. Taqqu, Eds., A Practical Guide to Heavy Tails: Statistical Techniques and Applications, Birkhäuser, Boston, Mass, USA, 1998.View at: MathSciNet
J. B. Bassingthwaighte, L. S. Liebovitch, and B. J. West, Fractal Physiology, Oxford University Press, Oxford, UK, 1994.
P. Doukhan, G. Oppenheim, and M. S. Taqqu, Eds., Theory and Applications of Long-Range Dependence, Birkhäuser, Boston, Mass, USA, 2003.View at: MathSciNet
M. Li, Teletraffic Modeling Relating to Generalized Cauchy Process: Empirical Study, VDM, Saarbrücken, Germany, 2009.
G. A. Korn and T. M. Korn, Mathematical Handbook for Scientists and Engineers, McGraw-Hill, Boston, Mass, USA, 1961.
S. S. Wolff, J. L. Gastwirth, and J. B. Thomas, “Linear optimum predictors,” IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 30–32, 1967.View at: Google Scholar
J. Markhoul, “Linear prediction: a tutorial review,” Proceedings of the IEEE, vol. 63, no. 4, pp. 561–580, 1975.View at: Google Scholar
V. D. Kudritskii, “Optimal linear extrapolation algorithm for realization of a vector random sequence observed without errors,” Cybernetics and Systems Analysis, vol. 34, no. 2, pp. 185–189, 1998.View at: Google Scholar
B. S. Atal, “The history of linear prediction,” IEEE Signal Processing Magazine, vol. 23, no. 2, pp. 154–161, 2006.View at: Google Scholar
H. Michiel and K. Laevens, “Teletraffic engineering in a broad-band era,” Proceedings of the IEEE, vol. 85, no. 12, pp. 2007–2032, 1997.View at: Google Scholar
D. H. Griffel, Applied Functional Analysis, Ellis Horwood Series in Mathematics and Its Application, Ellis Horwood, Chichester, UK, 1981.View at: MathSciNet
C. K. Liu, Applied Functional Analysis, Defence Industry Press, Beijing, China, 1986.
J. Beran, “Statistical methods for data with long-range dependence,” Statistical Science, vol. 7, no. 4, pp. 404–416, 1992.View at: Google Scholar
Z.-H. Liu, “Chaotic time series analysis,” to appear in Mathematical Problems in Engineering.View at: Google Scholar
C. Cattani, “Harmonic wavelet approximation of random, fractal and high frequency signals,” to appear in Telecommunication Systems.View at: Google Scholar
C. Cattani, “Harmonic wavelet analysis of a localized fractal,” International Journal of Engineering and Interdisciplinary Mathematics, vol. 1, no. 1, pp. 35–44, 2009.View at: Google Scholar