Table of Contents Author Guidelines Submit a Manuscript
Journal of Applied Mathematics and Stochastic Analysis
Volume 2006, Article ID 81593, 8 pages
http://dx.doi.org/10.1155/JAMSA/2006/81593

A semimartingale characterization of average optimal stationary policies for Markov decision processes

1Department of Mathematics, South China Normal University, Guangzhou 510631, China
2The School of Mathematics and Computational Science, Zhongshan University, Guangzhou 510275, China

Received 30 November 2004; Revised 10 June 2005; Accepted 22 June 2005

Copyright © 2006 Quanxin Zhu and Xianping Guo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. A. Arapostathis, V. S. Borkar, E. Fernández-Gaucherand, M. K. Ghosh, and S. I. Marcus, “Discrete-time controlled Markov processes with average cost criterion: a survey,” SIAM Journal on Control and Optimization, vol. 31, no. 2, pp. 282–344, 1993. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  2. C. Derman, Finite State Markovian Decision Processes, vol. 67 of Mathematics in Science and Engineering, Academic Press, New York, 1970. View at Zentralblatt MATH · View at MathSciNet
  3. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, vol. 235 of Fundamental Principles of Mathematical Sciences, Springer, Berlin, 1979. View at Zentralblatt MATH · View at MathSciNet
  4. E. Gordienko and O. Hernández-Lerma, “Average cost Markov control processes with weighted norms: existence of canonical policies,” Applicationes Mathematicae, vol. 23, no. 2, pp. 199–218, 1995. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  5. X. P. Guo, J. Y. Liu, and K. Liu, “Nonhomogeneous Markov decision processes with Borel state space—the average criterion with nonuniformly bounded rewards,” Mathematics of Operations Research, vol. 25, no. 4, pp. 667–678, 2000. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  6. X. P. Guo and P. Shi, “Limiting average criteria for nonstationary Markov decision processes,” SIAM Journal on Optimization, vol. 11, no. 4, pp. 1037–1053, 2001. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  7. X. P. Guo and Q. X. Zhu, “Average optimality for Markov decision processes in Borel spaces: a new condition and approach,” to appear in Journal of Applied Probability.
  8. O. Hernández-Lerma, Adaptive Markov Control Processes, vol. 79 of Applied Mathematical Sciences, Springer, New York, 1989. View at Zentralblatt MATH · View at MathSciNet
  9. O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes. Basic Optimality Criteria, vol. 30 of Applications of Mathematics (New York), Springer, New York, 1996. View at MathSciNet
  10. O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, vol. 42 of Applications of Mathematics (New York), Springer, New York, 1999. View at Zentralblatt MATH · View at MathSciNet
  11. R. A. Howard, Dynamic Programming and Markov Processes, John Wiley & Sons, New York, 1960. View at Zentralblatt MATH · View at MathSciNet
  12. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, John Wiley & Sons, New York, 1994. View at Zentralblatt MATH · View at MathSciNet
  13. L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems, Wiley Series in Probability and Statistics: Applied Probability and Statistics, John Wiley & Sons, New York, 1999. View at Zentralblatt MATH · View at MathSciNet
  14. L. I. Sennott, “Average reward optimization theory for denumerable state spaces,” in Handbook of Markov Decision Processes, E. A. Feinberg and A. Shwartz, Eds., vol. 40 of Internat. Ser. Oper. Res. Management Sci., pp. 153–172, Kluwer Academic, Massachusetts, 2002. View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  15. Q. X. Zhu and X. P. Guo, “Another set of condition for strong n(n=1,0) discount optimality in Markov decision processes,” Stochastic Analysis and Applications, vol. 23, no. 5, pp. 953–974, 2005. View at Publisher · View at Google Scholar · View at MathSciNet
  16. Q. X. Zhu and X. P. Guo, “Unbounded cost Markov decision processes with limsup and liminf average criteria: new conditions,” Mathematical Methods of Operations Research, vol. 61, no. 3, pp. 469–482, 2005. View at Publisher · View at Google Scholar