Journal of Applied Mathematics and Stochastic Analysis
Volume 2006, Article ID 81593, 8 pages
http://dx.doi.org/10.1155/JAMSA/2006/81593
A semimartingale characterization of average optimal stationary policies for Markov decision processes
1Department of Mathematics, South China Normal University, Guangzhou 510631, China
2The School of Mathematics and Computational Science, Zhongshan University, Guangzhou 510275, China
Received 30 November 2004; Revised 10 June 2005; Accepted 22 June 2005
Copyright © 2006 Quanxin Zhu and Xianping Guo. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
This paper deals with discrete-time Markov decision processes with Borel state and action spaces. The criterion to be minimized is the average expected costs, and the costs may have neither upper nor lower bounds. In our former paper (to appear in Journal of Applied Probability), weaker conditions are proposed to ensure the existence of average optimal stationary policies. In this paper, we further study some properties of optimal policies. Under these weaker conditions, we not only obtain two necessary and sufficient conditions for optimal policies, but also give a “semimartingale characterization” of an average optimal stationary policy.