Biomedical Signal Processing and Modeling Complexity of Living Systems 2013View this Special Issue
Research Article | Open Access
Heavy-Tailed Prediction Error: A Difficulty in Predicting Biomedical Signals of Noise Type
A fractal signal in biomedical engineering may be characterized by noise, that is, the power spectrum density (PSD) divergences at . According the Taqqu’s law, noise has the properties of long-range dependence and heavy-tailed probability density function (PDF). The contribution of this paper is to exhibit that the prediction error of a biomedical signal of noise type is long-range dependent (LRD). Thus, it is heavy-tailed and of noise. Consequently, the variance of the prediction error is usually large or may not exist, making predicting biomedical signals of noise type difficult.
Signals of noise type are widely observed in biomedical engineering, ranging from heart rate to DNA and protein, see, for example, [1–37], just to cite a few. Predicting such a type of signals is desired in the field [38–43]. A fundamental issue in this regard is whether a biomedical signal of noise type to be predicted is predicable or not.
The predictability of signals of non- noise type is well studied [44–48]. However, the predictability of noise is rarely reported, to our best knowledge. Since many phenomena in biomedical engineering are characterized by noise [1–37], the predictability issue of noise is worth investigating.
Note that minimizing the mean square error (MSE) of prediction is a commonly used criterion in both theory and practical techniques of prediction, see, for example, [49–68]. Therefore, a sufficient condition for a biomedical signal to be predictable is that the variance of its predication error exists. If the variance of the predication error does not exist, on the contrary, it may be difficult to be predicted if not unpredictable. In the case of a signal being bandlimited, the variance of its predication error is generally finite. Consequently, it may be minimized and it is predictable. However, that is not always the case for biomedical signals of noise type.
Let be a biomedical signal in the class of noise. Then, its PDF is heavy-tailed, and it is LRD, see, for example, Adler et al. , Samorodnitsky and Taqqu , Mandelbrot , Li and Zhao . Due to that, here and below, the terms, noise, LRD random function, and heavy-tailed random function are interchangeable.
Let be the PDF of a biomedical signal of noise type. Then, its variance is expressed by where is the mean of if it exists. The term of heavy tail in statistics implies that is large. Theoretically speaking, in general, we cannot assume that always exists . In some cases, such as the Pareto distribution, the Cauchy distribution, -stable distributions , may be infinite. That does not exist is particularly true for signals in biomedical engineering and physiology, see Bassingthwaighte et al.  for the interpretation of this point of view.
Recall that a prediction error is a random function as we shall soon mention below. Therefore, whether the prediction error is of noise, or equivalently, heavy-tailed, turns to be a crucial issue we need studying. We aim at, in this research, exhibiting that prediction error of noise is heavy-tailed and accordingly is of noise. Thus, generally speaking, the variance of a prediction error of a biomedical signal of noise type may not exist or large. That is a reason why predicting biomedical signals of noise type is difficult.
The rest of this paper is organized as follows. Heavy-tailed prediction errors occurring in the prediction of biomedical signals of noise type are explained in Section 2. Discussions are in Section 3, which is followed by conclusions.
2. Prediction Errors of Noise Type
We use to represent a biomedical signal in the discrete case for , where is the set of natural numbers. Let be a given sample of for . Denote by the predicted values of for . Then, the prediction error denoted by is given by
If one uses the given sample of for to obtain the predictions denoted by for , the error is usually different from (2), which implies that the error is a random variable. Denote by the PDF of . Then, its variance is expressed by where is the mean of .
Let be the operator of a predictor. Then, A natural requirement in terms of is that should be minimized. Thus, the premise that can be minimized is that it exists.
It is obviously seen that may be large if is heavy tailed. In a certain cases, may not exist. To explain the latter, we assume that follows a type of heavy-tailed distribution called the Pareto distribution.
Denote by the PDF of the Pareto distribution. Then , it is in the form where , , and . The mean and variance of are, respectively, expressed by The above exhibits that does not exist if or and if follows the Pareto distribution.
Note that the situation that does not exist may not occur if is light-tailed. Therefore, the question in this regard is whether is heavy-tailed if a biomedical signal is of noise. The answer to that question is affirmative. We explain it below.
Theorem 1. Let be a biomedical signal of noise type to be predicted. Then, its prediction error is heavy-tailed. Consequently, it is of noise.
Proof. Let be the autocorrelation function (ACF) of . Then,
where is lag and the mean operator. Let be the ACF of . Then,
Let be the ACF of . Then,
Note that In the above expression, is the cross-correlation between and . On the other side, is the cross-correlation between and . Since , we have
Recall that is noise. Thus, it is heavy-tailed and hence LRD. Consequently, for a constant , we have On the other hand, the predicted series is LRD. Thus, for a constant , the following holds:
In (11), if is summable, that is, it decays faster than or , it may be ignored for . In this case, is still non-summable. In fact, one has On the other side, when is non-summable, is non-summable too. In any case, we may write by Therefore, the prediction error is LRD. Its PDF is heavy-tailed according to the Taqqu’s law. Following , therefore, is a noise. This completes the proof.
The present result implies that cautions are needed for dealing with predication errors of biomedical signals of noise type. In fact, if specific biomedical signals are in the class of noise, the variances of their prediction errors may not exist or large . Tucker and Garway-Heath used to state that their prediction errors with either prediction model they used are large . The result in this paper may in a way provide their research with an explanation.
Due to the fact that a biomedical signal may be of noise, PDF estimation is suggested as a preparatory stage for prediction. As a matter of fact, if a PDF estimation of biomedical signal is light-tailed, its variance of prediction error exists. On the contrary, the variance of the prediction error may not exist. In the latter case, special techniques have to be considered [75–78]. For instance, weighting prediction error may be a technique necessarily to be taken into account, which is suggested in the domain of generalized functions over the Schwartz distributions .
We have explained that the prediction error in predicting biomedical signals of noise type is usually LRD. This implies that its PDF is heavy-tailed and noise. Consequently, may in general be large. In some cases , may not exist, making the prediction of biomedical signals of noise type difficult with the way of minimizing .
This work was supported in part by the 973 plan under the Project Grant no. 2011CB302800 and by the National Natural Science Foundation of China under the Project Grant no. 61272402, 61070214, and 60873264.
- N. Aoyagi, Z. R. Struzik, K. Kiyono, and Y. Yamamoto, “Autonomic imbalance induced breakdown of long-range dependence in healthy heart rate,” Methods of Information in Medicine, vol. 46, no. 2, pp. 174–178, 2007.
- S. Tong, D. Jiang, Z. Wang, Y. Zhu, R. G. Geocadin, and N. V. Thakor, “Long range correlations in the heart rate variability following the injury of cardiac arrest,” Physica A, vol. 380, no. 1-2, pp. 250–258, 2007.
- N. V. Sarlis, E. S. Skordas, and P. A. Varotsos, “Heart rate variability in natural time and ‘noise’,” Europhysics Letters, vol. 87, no. 1, Article ID 18003, 2009.
- Z. R. Struzik, J. Hayano, R. Soma, S. Kwak, and Y. Yamamoto, “Aging of complex heart rate dynamics,” IEEE Transactions on Biomedical Engineering, vol. 53, no. 1, pp. 89–94, 2006.
- U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim, and J. S. Suri, “Heart rate variability: a review,” Medical and Biological Engineering and Computing, vol. 44, no. 12, pp. 1031–1051, 2006.
- J. H. T. Bates, G. N. Maksym, D. Navajas, and B. Suki, “Lung tissue rheology and noise,” Annals of Biomedical Engineering, vol. 22, no. 6, pp. 674–681, 1994.
- J. M. Halley and W. E. Kunin, “Extinction risk and the family of noise models,” Theoretical Population Biology, vol. 56, no. 3, pp. 215–230, 1999.
- M. C. Wichmann, K. Johst, M. Schwager, B. Blasius, and F. Jeltsch, “Extinction risk, coloured noise and the scaling of variance,” Theoretical Population Biology, vol. 68, no. 1, pp. 29–40, 2005.
- Z. Yang, L. Hoang, Q. Zhao, E. Keefer, and W. Liu, “ neural noise reduction and spike feature extraction using a subset of informative samples,” Annals of Biomedical Engineering, vol. 39, no. 4, pp. 1264–1277, 2011.
- J. Ruseckas and B. Kaulakys, “Tsallis distributions and noise from nonlinear stochastic differential equations,” Physical Review E, vol. 84, no. 5, Article ID 051125, 7 pages, 2011.
- F. Beckers, B. Verheyden, and A. E. Aubert, “Aging and nonlinear heart rate control in a healthy population,” American Journal of Physiology, vol. 290, no. 6, pp. H2560–H2570, 2006.
- B. Pilgram and D. T. Kaplan, “Nonstationarity and noise characteristics in heart rate,” American Journal of Physiology, vol. 276, no. 1, pp. R1–R9, 1999.
- P. Szendro, G. Vincze, and A. Szasz, “Pink-noise behaviour of biosystems,” European Biophysics Journal, vol. 30, no. 3, pp. 227–231, 2001.
- G. Massiera, K. M. Van Citters, P. L. Biancaniello, and J. C. Crocker, “Mechanics of single cells: rheology, time dependence, and fluctuations,” Biophysical Journal, vol. 93, no. 10, pp. 3703–3713, 2007.
- Y. Murase, T. Shimada, N. Ito, and P. A. Rikvold, “Effects of demographic stochasticity on biological community assembly on evolutionary time scales,” Physical Review E, vol. 81, no. 4, Article ID 041908, 14 pages, 2010.
- T. Yokogawa and T. Harada, “Generality of a power-law long-term correlation in beat timings of single cardiac cells,” Biochemical and Biophysical Research Communications, vol. 387, no. 1, pp. 19–24, 2009.
- T. Harada, T. Yokogawa, T. Miyaguchi, and H. Kori, “Singular behavior of slow dynamics of single excitable cells,” Biophysical Journal, vol. 96, no. 1, pp. 255–267, 2009.
- A. Eke, P. Hermán, J. B. Bassingthwaighte et al., “Physiological time series: distinguishing fractal noises from motions,” Pflugers Archiv, vol. 439, no. 4, pp. 403–415, 2000.
- B. J. West, “Fractal physiology and the fractional calculus: a perspective,” Frontiers in Fractal Physiology, vol. 1, article 12, 2010.
- P. Grigolini, G. Aquino, M. Bologna, M. Luković, and B. J. West, “A theory of noise in human cognition,” Physica A, vol. 388, no. 19, pp. 4192–4204, 2009.
- F. Grüneis, M. Nakao, Y. Mizutani, M. Yamamoto, M. Meesmann, and T. Musha, “Further study on fluctuations observed in central single neurons during REM sleep,” Biological Cybernetics, vol. 68, no. 3, pp. 193–198, 1993.
- H. Sheng, Y.-Q. Chen, and T.-S. Qiu, “Heavy-tailed distribution and local long memory in time series of molecular motion on the cell membrane,” Fluctuation and Noise Letters, vol. 10, no. 1, pp. 93–119, 2011.
- B. J. West and W. Deering, “Fractal physiology for physicists: Lévy statistics,” Physics Report, vol. 246, no. 1-2, pp. 1–100, 1994.
- W. Deering and B. J. West, “Fractal physiology,” IEEE Engineering in Medicine and Biology Magazine, vol. 11, no. 2, pp. 40–46, 1992.
- B. J. West, “Physiology in fractal dimensions: error tolerance,” Annals of Biomedical Engineering, vol. 18, no. 2, pp. 135–149, 1990.
- M. Joyeux, S. Buyukdagli, and M. Sanrey, “ Fluctuations of DNA temperature at thermal denaturation,” Physical Review E, vol. 75, no. 6, Article ID 061914, 9 pages, 2007.
- C. Cattani, “Fractals and hidden symmetries in DNA,” Mathematical Problems in Engineering, vol. 2010, Article ID 507056, 31 pages, 2010.
- C. Cattani, E. Laserra, and I. Bochicchio, “Simplicial approach to fractal structures,” Mathematical Problems in Engineering, vol. 2012, Article ID 958101, 21 pages, 2012.
- P. Herman and A. Eke, “Nonlinear analysis of blood cell flux fluctuations in the rat brain cortex during stepwise hypotension challenge,” Journal of Cerebral Blood Flow & Metabolism, vol. 26, no. 9, pp. 1189–1197, 2006.
- M. Baumert, V. Baier, and A. Voss, “Long-term correlations and fractal dimension of beat-to-beat blood pressure dynamics,” Fluctuation and Noise Letters, vol. 5, no. 4, pp. L549–L555, 2005.
- C. Cattani, “On the existence of wavelet symmetries in Archaea DNA,” Computational and Mathematical Methods in Medicine, vol. 2012, Article ID 673934, 21 pages, 2012.
- S. Y. Ponomarev, V. Putkaradze, and T. C. Bishop, “Relaxation dynamics of nucleosomal DNA,” Physical Chemistry Chemical Physics, vol. 11, no. 45, pp. 10633–10643, 2009.
- J. B. Bassingthwaighte, L. S. Liebovitch, and B. J. West, Fractal Physiology, Oxford University Press, 1994.
- D. Craciun, A. Isvoran, and N. M. Avram, “Long range correlation of hydrophilicity and flexibility along the calcium binding protein chains,” Physica A, vol. 388, no. 21, pp. 4609–4618, 2009.
- J. Siódmiak, J. J. Uher, I. Santamaría-Holek, N. Kruszewska, and A. Gadomski, “On the protein crystal formation as an interface-controlled process with prototype ion-channeling effect,” Journal of Biological Physics, vol. 33, no. 4, pp. 313–329, 2007.
- S. C. Kou and X. S. Xie, “Generalized langevin equation with fractional gaussian noise: subdiffusion within a single protein molecule,” Physical Review Letters, vol. 93, no. 18, Article ID 180603, 4 pages, 2004.
- H. Sheng, Y.-Q. Chen, and T.-S. Qiu, Fractional Processes and Fractional Order Signal Processing, Springer, 2012.
- M. Panella, “Advances in biological time series prediction by neural networks,” Biomedical Signal Processing and Control, vol. 6, no. 2, pp. 112–120, 2011.
- Y.-R. Cho and A. Zhang, “Predicting protein function by frequent functional association pattern mining in protein interaction networks,” IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 1, pp. 30–36, 2010.
- A. Castro, M. A. L. Marques, D. Varsano, F. Sottile, and A. Rubio, “The challenge of predicting optical properties of biomolecules: what can we learn from time-dependent density-functional theory?” Comptes Rendus Physique, vol. 10, no. 6, pp. 469–490, 2009.
- Q. Lü, H. J. Wu, J. Z. Wu et al., “A parallel ant colonies approach to de novo prediction of protein backbone in CASP8/9,” Science China Information Sciences. In press.
- B. R. Yang, W. Qu, L. J. Wang, and Y. Zhou, “A new intelligent prediction system model-the compound pyramid model,” Science China Information Sciences, vol. 55, no. 3, pp. 723–736, 2012.
- J. L. Suo, X. Y. Ji, and Q. H. Dai, “An overview of computational photography,” Science China Information Sciences, vol. 55, no. 6, pp. 1229–1248, 2012.
- A. Papoulis, “A note on the predictability of band-limited processes,” Proceedings of the IEEE, vol. 73, no. 8, pp. 1332–1333, 1985.
- S. Y. Chen, C. Y. Yao, G. Xiao, Y. S. Ying, and W. L. Wang, “Fault detection and prediction of clocks and timers based on computer audition and probabilistic neural networks,” in Proceedings of the 8th International Workshop on Artificial Neural Networks, IWANN 2005: Computational Intelligence and Bioinspired Systems, vol. 3512 of Lecture Notes in Computer Science, pp. 952–959, June 2005.
- R. J. Lyman, W. W. Edmonson, S. McCullough, and M. Rao, “The predictability of continuous-time, bandlimited processes,” IEEE Transactions on Signal Processing, vol. 48, no. 2, pp. 311–316, 2000.
- R. J. Lyman and W. W. Edmonson, “Linear prediction of bandlimited processes with flat spectral densities,” IEEE Transactions on Signal Processing, vol. 49, no. 7, pp. 1564–1569, 2001.
- N. Dokuchaev, “The predictability of band-limited, high-frequency and mixed processes in the presence of ideal low-pass filters,” Journal of Physics A, vol. 41, no. 38, Article ID 382002, 7 pages, 2008.
- N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series, John Wiley & Sons, 1964.
- A. N. Kolmogorov, “Interpolation and extrapolation of stationary random sequences,” Izvestiya Akademii Nauk SSSR, vol. 5, pp. 3–14, 1941.
- L. A. Zadeh and J. R. Ragazzini, “An extension of Wiener's theory of prediction,” Journal of Applied Physics, vol. 21, no. 7, pp. 645–655, 1950.
- R. J. Bhansali, “Asymptotic properties of the Wiener-Kolmogorov predictor. I,” Journal of the Royal Statistical Society B, vol. 36, no. 1, pp. 61–73, 1974.
- N. Levinson, “A heuristic exposition of Wiener’s mathematical theory of prediction and filtering,” Journal of Mathematical Physics, vol. 26, pp. 110–119, 1947.
- N. Levinson, “The Wiener RMS (root mean squares) error criterion in filter design and prediction,” Journal of Mathematical Physics, vol. 25, pp. 261–278, 1947.
- R. J. Bhansali, “Asymptotic mean-square error of predicting more than one-step ahead using the regression method,” Journal of the Royal Statistical Society C, vol. 23, no. 1, pp. 35–42, 1974.
- J. Makhoul, “Linear prediction: a tutorial review,” Proceedings of the IEEE, vol. 63, no. 4, pp. 561–580, 1975.
- D. L. Zimmerman and N. Cressie, “Mean squared prediction error in the spatial linear model with estimated covariance parameters,” Annals of the Institute of Statistical Mathematics, vol. 44, no. 1, pp. 27–43, 1992.
- D. Huang, “Levinson-type recursive algorithms for least-squares autoregression,” Journal of Time Series Analysis, vol. 11, no. 4, pp. 295–315, 2008.
- R. S. Deo, “Improved forecasting of autoregressive series by weighted least squares approximate REML estimation,” International Journal of Forecasting, vol. 28, no. 1, pp. 39–43, 2012.
- A. Rodríguez and E. Ruiz, “Bootstrap prediction mean squared errors of unobserved states based on the Kalman filter with estimated parameters,” Computational Statistics & Data Analysis, vol. 56, no. 1, pp. 62–74, 2012.
- M. Abt, “Estimating the prediction mean squared error in gaussian stochastic processes with exponential correlation structure,” Scandinavian Journal of Statistics, vol. 26, no. 4, pp. 563–578, 1999.
- R. Kohn and C. F. Ansley, “Prediction mean squared error for state space models with estimated parameters,” Biometrika, vol. 73, no. 2, pp. 467–473, 1986.
- R. T. Baillie, “Asymptotic prediction mean squared error for vector autoregressive models,” Biometrika, vol. 66, no. 3, pp. 675–678, 1979.
- P. Neelamegam, A. Jamaludeen, and A. Rajendran, “Prediction of calcium concentration in human blood serum using an artificial neural network,” Measurement, vol. 44, no. 2, pp. 312–319, 2011.
- E. S. G. Carotti, J. C. De Martin, R. Merletti, and D. Farina, “Compression of multidimensional biomedical signals with spatial and temporal codebook-excited linear prediction,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 11, pp. 2604–2610, 2009.
- W. Bachta, P. Renaud, L. Cuvillon, E. Laroche, A. Forgione, and J. Gangloff, “Motion prediction for computer-assisted beating heart surgery,” IEEE Transactions on Biomedical Engineering, vol. 56, no. 11, pp. 2551–2563, 2009.
- H.-H. Lin, C. L. Beck, and M. J. Bloom, “On the use of multivariable piecewise-linear models for predicting human response to anesthesia,” IEEE Transactions on Biomedical Engineering, vol. 51, no. 11, pp. 1876–1887, 2004.
- B. S. Atal, “The history of linear prediction,” IEEE Signal Processing Magazine, vol. 23, no. 2, pp. 154–161, 2006.
- R. J. Adler, R. E. Feldman, and M. S. Taqqu, Eds., A Practical Guide to Heavy Tails: Statistical Techniques and Applications, Birkhäuser, Boston, Mass, USA, 1998.
- G. Samorodnitsky and M. S. Taqqu, Stable Non-Gaussian Random Processes, Chapman & Hall, New York, NY, USA, 1994.
- B. B. Mandelbrot, Multifractals and 1/f Noise, Springer, 1998.
- M. Li and W. Zhao, “On noise,” Mathematical Problems in Engineering. In press.
- G. A. Korn and T. M. Korn, Mathematical Handbook for Scientists and Engineers, McGraw-Hill, 1961.
- A. Tucker and D. Garway-Heath, “The pseudotemporal bootstrap for predicting glaucoma from cross-sectional visual field data,” IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 1, pp. 79–85, 2010.
- M. Carlini and S. Castellucci, “Modelling the vertical heat exchanger in thermal basin,” in Proceedings of the International Conference on Computational Science and Its Applications (ICCSA '11), vol. 6785 of Lecture Notes in Computer Science, pp. 277–286, Springer.
- M. Carlini, C. Cattani, and A. Tucci, “Optical modelling of square solar concentrator,” in Proceedings of the International Conference on Computational Science and Its Applications (ICCSA '11), vol. 6785 of Lecture Notes in Computer Science, pp. 287–295, Springer.
- R. J. Bhansali and P. S. Kokoszka, “Prediction of long-memory time series: a tutorial review,” Lecture Notes in Physics, vol. 621, pp. 3–21, 2003.
- L. Bisaglia and S. Bordignon, “Mean square prediction error for long-memory processes,” Statistical Papers, vol. 43, no. 2, pp. 161–175, 2002.
- M. Li and J.-Y. Li, “On the predictability of long-range dependent series,” Mathematical Problems in Engineering, vol. 2010, Article ID 397454, 9 pages, 2010.
Copyright © 2012 Ming Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.