Biomedical Signal Processing and Modeling Complexity of Living Systems 2013View this Special Issue
Heavy-Tailed Prediction Error: A Difficulty in Predicting Biomedical Signals of Noise Type
A fractal signal in biomedical engineering may be characterized by noise, that is, the power spectrum density (PSD) divergences at . According the Taqqu’s law, noise has the properties of long-range dependence and heavy-tailed probability density function (PDF). The contribution of this paper is to exhibit that the prediction error of a biomedical signal of noise type is long-range dependent (LRD). Thus, it is heavy-tailed and of noise. Consequently, the variance of the prediction error is usually large or may not exist, making predicting biomedical signals of noise type difficult.
Signals of noise type are widely observed in biomedical engineering, ranging from heart rate to DNA and protein, see, for example, [1–37], just to cite a few. Predicting such a type of signals is desired in the field [38–43]. A fundamental issue in this regard is whether a biomedical signal of noise type to be predicted is predicable or not.
The predictability of signals of non- noise type is well studied [44–48]. However, the predictability of noise is rarely reported, to our best knowledge. Since many phenomena in biomedical engineering are characterized by noise [1–37], the predictability issue of noise is worth investigating.
Note that minimizing the mean square error (MSE) of prediction is a commonly used criterion in both theory and practical techniques of prediction, see, for example, [49–68]. Therefore, a sufficient condition for a biomedical signal to be predictable is that the variance of its predication error exists. If the variance of the predication error does not exist, on the contrary, it may be difficult to be predicted if not unpredictable. In the case of a signal being bandlimited, the variance of its predication error is generally finite. Consequently, it may be minimized and it is predictable. However, that is not always the case for biomedical signals of noise type.
Let be a biomedical signal in the class of noise. Then, its PDF is heavy-tailed, and it is LRD, see, for example, Adler et al. , Samorodnitsky and Taqqu , Mandelbrot , Li and Zhao . Due to that, here and below, the terms, noise, LRD random function, and heavy-tailed random function are interchangeable.
Let be the PDF of a biomedical signal of noise type. Then, its variance is expressed by where is the mean of if it exists. The term of heavy tail in statistics implies that is large. Theoretically speaking, in general, we cannot assume that always exists . In some cases, such as the Pareto distribution, the Cauchy distribution, -stable distributions , may be infinite. That does not exist is particularly true for signals in biomedical engineering and physiology, see Bassingthwaighte et al.  for the interpretation of this point of view.
Recall that a prediction error is a random function as we shall soon mention below. Therefore, whether the prediction error is of noise, or equivalently, heavy-tailed, turns to be a crucial issue we need studying. We aim at, in this research, exhibiting that prediction error of noise is heavy-tailed and accordingly is of noise. Thus, generally speaking, the variance of a prediction error of a biomedical signal of noise type may not exist or large. That is a reason why predicting biomedical signals of noise type is difficult.
The rest of this paper is organized as follows. Heavy-tailed prediction errors occurring in the prediction of biomedical signals of noise type are explained in Section 2. Discussions are in Section 3, which is followed by conclusions.
2. Prediction Errors of Noise Type
We use to represent a biomedical signal in the discrete case for , where is the set of natural numbers. Let be a given sample of for . Denote by the predicted values of for . Then, the prediction error denoted by is given by
If one uses the given sample of for to obtain the predictions denoted by for , the error is usually different from (2), which implies that the error is a random variable. Denote by the PDF of . Then, its variance is expressed by where is the mean of .
Let be the operator of a predictor. Then, A natural requirement in terms of is that should be minimized. Thus, the premise that can be minimized is that it exists.
It is obviously seen that may be large if is heavy tailed. In a certain cases, may not exist. To explain the latter, we assume that follows a type of heavy-tailed distribution called the Pareto distribution.
Denote by the PDF of the Pareto distribution. Then , it is in the form where , , and . The mean and variance of are, respectively, expressed by The above exhibits that does not exist if or and if follows the Pareto distribution.
Note that the situation that does not exist may not occur if is light-tailed. Therefore, the question in this regard is whether is heavy-tailed if a biomedical signal is of noise. The answer to that question is affirmative. We explain it below.
Theorem 1. Let be a biomedical signal of noise type to be predicted. Then, its prediction error is heavy-tailed. Consequently, it is of noise.
Proof. Let be the autocorrelation function (ACF) of . Then,
where is lag and the mean operator. Let be the ACF of . Then,
Let be the ACF of . Then,
Note that In the above expression, is the cross-correlation between and . On the other side, is the cross-correlation between and . Since , we have
Recall that is noise. Thus, it is heavy-tailed and hence LRD. Consequently, for a constant , we have On the other hand, the predicted series is LRD. Thus, for a constant , the following holds:
In (11), if is summable, that is, it decays faster than or , it may be ignored for . In this case, is still non-summable. In fact, one has On the other side, when is non-summable, is non-summable too. In any case, we may write by Therefore, the prediction error is LRD. Its PDF is heavy-tailed according to the Taqqu’s law. Following , therefore, is a noise. This completes the proof.
The present result implies that cautions are needed for dealing with predication errors of biomedical signals of noise type. In fact, if specific biomedical signals are in the class of noise, the variances of their prediction errors may not exist or large . Tucker and Garway-Heath used to state that their prediction errors with either prediction model they used are large . The result in this paper may in a way provide their research with an explanation.
Due to the fact that a biomedical signal may be of noise, PDF estimation is suggested as a preparatory stage for prediction. As a matter of fact, if a PDF estimation of biomedical signal is light-tailed, its variance of prediction error exists. On the contrary, the variance of the prediction error may not exist. In the latter case, special techniques have to be considered [75–78]. For instance, weighting prediction error may be a technique necessarily to be taken into account, which is suggested in the domain of generalized functions over the Schwartz distributions .
We have explained that the prediction error in predicting biomedical signals of noise type is usually LRD. This implies that its PDF is heavy-tailed and noise. Consequently, may in general be large. In some cases , may not exist, making the prediction of biomedical signals of noise type difficult with the way of minimizing .
This work was supported in part by the 973 plan under the Project Grant no. 2011CB302800 and by the National Natural Science Foundation of China under the Project Grant no. 61272402, 61070214, and 60873264.
N. Aoyagi, Z. R. Struzik, K. Kiyono, and Y. Yamamoto, “Autonomic imbalance induced breakdown of long-range dependence in healthy heart rate,” Methods of Information in Medicine, vol. 46, no. 2, pp. 174–178, 2007.View at: Google Scholar
J. Ruseckas and B. Kaulakys, “Tsallis distributions and noise from nonlinear stochastic differential equations,” Physical Review E, vol. 84, no. 5, Article ID 051125, 7 pages, 2011.View at: Google Scholar
B. Pilgram and D. T. Kaplan, “Nonstationarity and noise characteristics in heart rate,” American Journal of Physiology, vol. 276, no. 1, pp. R1–R9, 1999.View at: Google Scholar
B. J. West and W. Deering, “Fractal physiology for physicists: Lévy statistics,” Physics Report, vol. 246, no. 1-2, pp. 1–100, 1994.View at: Google Scholar
J. B. Bassingthwaighte, L. S. Liebovitch, and B. J. West, Fractal Physiology, Oxford University Press, 1994.
H. Sheng, Y.-Q. Chen, and T.-S. Qiu, Fractional Processes and Fractional Order Signal Processing, Springer, 2012.
Q. Lü, H. J. Wu, J. Z. Wu et al., “A parallel ant colonies approach to de novo prediction of protein backbone in CASP8/9,” Science China Information Sciences. In press.View at: Google Scholar
B. R. Yang, W. Qu, L. J. Wang, and Y. Zhou, “A new intelligent prediction system model-the compound pyramid model,” Science China Information Sciences, vol. 55, no. 3, pp. 723–736, 2012.View at: Google Scholar
J. L. Suo, X. Y. Ji, and Q. H. Dai, “An overview of computational photography,” Science China Information Sciences, vol. 55, no. 6, pp. 1229–1248, 2012.View at: Google Scholar
A. Papoulis, “A note on the predictability of band-limited processes,” Proceedings of the IEEE, vol. 73, no. 8, pp. 1332–1333, 1985.View at: Google Scholar
S. Y. Chen, C. Y. Yao, G. Xiao, Y. S. Ying, and W. L. Wang, “Fault detection and prediction of clocks and timers based on computer audition and probabilistic neural networks,” in Proceedings of the 8th International Workshop on Artificial Neural Networks, IWANN 2005: Computational Intelligence and Bioinspired Systems, vol. 3512 of Lecture Notes in Computer Science, pp. 952–959, June 2005.View at: Google Scholar
R. J. Lyman, W. W. Edmonson, S. McCullough, and M. Rao, “The predictability of continuous-time, bandlimited processes,” IEEE Transactions on Signal Processing, vol. 48, no. 2, pp. 311–316, 2000.View at: Google Scholar
N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, John Wiley & Sons, 1964.
A. N. Kolmogorov, “Interpolation and extrapolation of stationary random sequences,” Izvestiya Akademii Nauk SSSR, vol. 5, pp. 3–14, 1941.View at: Google Scholar
R. J. Bhansali, “Asymptotic properties of the Wiener-Kolmogorov predictor. I,” Journal of the Royal Statistical Society B, vol. 36, no. 1, pp. 61–73, 1974.View at: Google Scholar
N. Levinson, “A heuristic exposition of Wiener’s mathematical theory of prediction and filtering,” Journal of Mathematical Physics, vol. 26, pp. 110–119, 1947.View at: Google Scholar
N. Levinson, “The Wiener RMS (root mean squares) error criterion in filter design and prediction,” Journal of Mathematical Physics, vol. 25, pp. 261–278, 1947.View at: Google Scholar
R. J. Bhansali, “Asymptotic mean-square error of predicting more than one-step ahead using the regression method,” Journal of the Royal Statistical Society C, vol. 23, no. 1, pp. 35–42, 1974.View at: Google Scholar
J. Makhoul, “Linear prediction: a tutorial review,” Proceedings of the IEEE, vol. 63, no. 4, pp. 561–580, 1975.View at: Google Scholar
D. Huang, “Levinson-type recursive algorithms for least-squares autoregression,” Journal of Time Series Analysis, vol. 11, no. 4, pp. 295–315, 2008.View at: Google Scholar
M. Abt, “Estimating the prediction mean squared error in gaussian stochastic processes with exponential correlation structure,” Scandinavian Journal of Statistics, vol. 26, no. 4, pp. 563–578, 1999.View at: Google Scholar
R. T. Baillie, “Asymptotic prediction mean squared error for vector autoregressive models,” Biometrika, vol. 66, no. 3, pp. 675–678, 1979.View at: Google Scholar
E. S. G. Carotti, J. C. De Martin, R. Merletti, and D. Farina, “Compression of multidimensional biomedical signals with spatial and temporal codebook-excited linear prediction,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 11, pp. 2604–2610, 2009.View at: Publisher Site | Google Scholar
B. S. Atal, “The history of linear prediction,” IEEE Signal Processing Magazine, vol. 23, no. 2, pp. 154–161, 2006.View at: Google Scholar
R. J. Adler, R. E. Feldman, and M. S. Taqqu, Eds., A Practical Guide to Heavy Tails: Statistical Techniques and Applications, Birkhäuser, Boston, Mass, USA, 1998.
G. Samorodnitsky and M. S. Taqqu, Stable Non-Gaussian Random Processes, Chapman & Hall, New York, NY, USA, 1994.
B. B. Mandelbrot, Multifractals and 1/f Noise, Springer, 1998.
M. Li and W. Zhao, “On noise,” Mathematical Problems in Engineering. In press.View at: Google Scholar
G. A. Korn and T. M. Korn, Mathematical Handbook for Scientists and Engineers, McGraw-Hill, 1961.
M. Carlini and S. Castellucci, “Modelling the vertical heat exchanger in thermal basin,” in Proceedings of the International Conference on Computational Science and Its Applications (ICCSA '11), vol. 6785 of Lecture Notes in Computer Science, pp. 277–286, Springer.View at: Google Scholar
M. Carlini, C. Cattani, and A. Tucci, “Optical modelling of square solar concentrator,” in Proceedings of the International Conference on Computational Science and Its Applications (ICCSA '11), vol. 6785 of Lecture Notes in Computer Science, pp. 287–295, Springer.View at: Google Scholar
R. J. Bhansali and P. S. Kokoszka, “Prediction of long-memory time series: a tutorial review,” Lecture Notes in Physics, vol. 621, pp. 3–21, 2003.View at: Google Scholar