`Journal of Applied MathematicsVolume 2008, Article ID 639145, 10 pageshttp://dx.doi.org/10.1155/2008/639145`
Research Article

## A Strong Limit Theorem for Functions of Continuous Random Variables and an Extension of the Shannon-McMillan Theorem

1School of Finance and Statistics, East China Normal University, Shanghai 200241, China
2College of Applied Sciences, Beijing University of Technology, Beijing 100022, China
3School of Sciences, Hebei University of Technology, Tianjin 300130, China
4College of Mathematics and Science, Luoyang Normal University, Henan 471022, China

Received 1 November 2007; Revised 10 January 2008; Accepted 29 March 2008

Copyright © 2008 Gaorong Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. R. G. Laha and V. K. Rohatgi, Probability Theory, Wiley Series in Probability and Mathematical Statistic, John Wiley & Sons, New York, NY, USA, 1979.
2. P. Billingsley, Probability and Measure, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York, NY, USA, 2nd edition, 1986.
3. W. Liu, “Relative entropy densities and a class of limit theorems of the sequence of $m$-valued random variables,” The Annals of Probability, vol. 18, no. 2, pp. 829–839, 1990.
4. W. Liu, “A class of strong deviation theorems and Laplace transform methods,” Chinese Science Bulletin, vol. 43, no. 10, pp. 1036–1041, 1998.
5. W. Liu and Y. Wang, “A strong limit theorem expressed by inequalities for the sequences of absolutely continuous random variables,” Hiroshima Mathematical Journal, vol. 32, no. 3, pp. 379–387, 2002.
6. W. Yang, “Some limit properties for Markov chains indexed by a homogeneous tree,” Statistics & Probability Letters, vol. 65, no. 3, pp. 241–250, 2003.
7. W. Liu, Strong Deviation Theorems and Analytic Method, Science Press, Beijing, China, 2003.
8. J. L. Doob, Stochastic Processes, John Wiley & Sons, New York, NY, USA, 1953.
9. W. Liu and J. Wang, “A strong limit theorem on gambling systems,” Journal of Multivariate Analysis, vol. 84, no. 2, pp. 262–273, 2003.
10. T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley Series in Telecommunications, John Wiley & Sons, New York, NY, USA, 1991.
11. C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, 623–656, 1948.
12. P. H. Algoet and T. M. Cover, “A sandwich proof of the Shannon-McMillan-Breiman theorem,” The Annals of Probability, vol. 16, no. 2, pp. 899–909, 1988.
13. A. R. Barron, “The strong ergodic theorem for densities: generalized Shannon-McMillan-Breiman theorem,” The Annals of Probability, vol. 13, no. 4, pp. 1292–1303, 1985.
14. K. L. Chung, “A note on the ergodic theorem of information theory,” The Annals of Mathematical Statistics, vol. 32, no. 2, pp. 612–614, 1961.
15. J. C. Kieffer, “A simple proof of the Moy-Perez generalization of the Shannon-McMillan theorem,” Pacific Journal of Mathematics, vol. 51, pp. 203–206, 1974.
16. J. C. Kieffer, “A counterexample to Perez's generalization of the Shannon-McMillan theorem,” The Annals of Probability, vol. 1, no. 2, pp. 362–364, 1973.
17. B. McMillan, “The basic theorems of information theory,” The Annals of Mathematical Statistics, vol. 24, no. 2, pp. 196–219, 1953.
18. W. Liu and W. Yang, “An extension of Shannon-McMillan theorem and some limit properties for nonhomogeneous Markov chains,” Stochastic Processes and Their Applications, vol. 61, no. 1, pp. 129–145, 1996.
19. W. Liu and W. Yang, “The Markov approximation of the sequences of $N$-valued random variables and a class of small deviation theorems,” Stochastic Processes and Their Applications, vol. 89, no. 1, pp. 117–130, 2000.
20. R. M. Gray, Entropy and Information Theory, Springer, New York, NY, USA, 1990.