Table of Contents Author Guidelines Submit a Manuscript
Journal of Applied Mathematics
Volume 2012, Article ID 103698, 16 pages
http://dx.doi.org/10.1155/2012/103698
Research Article

A Version of the Euler Equation in Discounted Markov Decision Processes

Facultad de Ciencias Físico Matemáticas, Benemérita Universidad Autónoma de Puebla, Avenida San Claudio y Río Verde, Col. San Manuel, CU, 72570 Puebla, PUE, Mexico

Received 21 July 2012; Revised 29 August 2012; Accepted 6 September 2012

Academic Editor: Vu Phat

Copyright © 2012 H. Cruz-Suárez et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. D. P. Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice-Hall, Belmont, Tenn, USA, 1987.
  2. E. B. Dynkin and A. A. Yushkevich, Controlled Markov Processes, vol. 235, Springer, New York, NY, USA, 1979.
  3. O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, vol. 30, Springer, New York, NY, USA, 1996.
  4. O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, vol. 42, Springer, New York, NY, USA, 1999.
  5. N. Stokey, R. Lucas, and E. Prescott, Recursive Methods in Economic Dynamics, Harvard University Press, Cambridge, UK, 1989.
  6. K. J. Arrow, “A note on uncertainty and discounting in models of economic growth,” Journal of Risk and Uncertainty, vol. 38, no. 2, pp. 87–94, 2009. View at Google Scholar
  7. W. A. Brock and L. J. Mirman, “Optimal economic growth and uncertainty: the discounted case,” Journal of Economic Theory, vol. 4, no. 3, pp. 479–513, 1972. View at Publisher · View at Google Scholar
  8. H. Cruz-Suárez, R. Montes-de-Oca, and G. Zacarías, “A consumption-investment problem modelled as a discounted Markov decision process,” Kybernetika, vol. 47, no. 6, pp. 740–760, 2011. View at Google Scholar
  9. T. Kamihigashi, “Stochastic optimal growth with bounded or unbounded utility and with bounded or unbounded shocks,” Journal of Mathematical Economics, vol. 43, no. 3-4, pp. 477–500, 2007. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  10. I. Karatzas and W. D. Sudderth, “Two characterizations of optimality in dynamic programming,” Applied Mathematics and Optimization, vol. 61, no. 3, pp. 421–434, 2010. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  11. A. Jaśkiewicz and A. S. Nowak, “Discounted dynamic programming with unbounded returns: application to economic models,” Journal of Mathematical Analysis and Applications, vol. 378, no. 2, pp. 450–462, 2011. View at Publisher · View at Google Scholar
  12. D. Levhari and T. N. Srinivasan, “Optimal savings under uncertainty,” Review of Economic Studies, vol. 36, pp. 153–163, 1969. View at Google Scholar
  13. L. J. Mirman and I. Zilcha, “On optimal growth under uncertainty,” Journal of Economic Theory, vol. 2, no. 3, pp. 329–339, 1975. View at Google Scholar · View at Zentralblatt MATH
  14. H. Cruz-Suárez and R. Montes-de-Oca, “Discounted Markov control processes induced by deterministic systems,” Kybernetika, vol. 42, no. 6, pp. 647–664, 2006. View at Google Scholar
  15. L. M. Benveniste and J. A. Scheinkman, “On the differentiability of the value function in dynamic models of economics,” Econometrica, vol. 47, no. 3, pp. 727–732, 1979. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  16. H. Cruz-Suárez and R. Montes-de-Oca, “An envelope theorem and some applications to discounted Markov decision processes,” Mathematical Methods of Operations Research, vol. 67, no. 2, pp. 299–321, 2008. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  17. P. Milgrom and I. Segal, “Envelope theorems for arbitrary choice sets,” Econometrica. Journal of the Econometric Society, vol. 70, no. 2, pp. 583–601, 2002. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  18. X.-R. Cao, Stochastic Learning and Optimization, Springer, New York, NY, USA, 2007. View at Publisher · View at Google Scholar
  19. R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, USA, 1970.
  20. D. Cruz-Suárez, R. Montes-de-Oca, and F. Salem-Silva, “Conditions for the uniqueness of optimal policies of discounted Markov decision processes,” Mathematical Methods of Operations Research, vol. 60, no. 3, pp. 415–436, 2004. View at Publisher · View at Google Scholar · View at Zentralblatt MATH
  21. A. De la Fuente, Mathematical Methods and Models for Economists, Cambridge University Press, Cambridge, UK, 2000.