Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015 (2015), Article ID 350102, 9 pages
http://dx.doi.org/10.1155/2015/350102
Research Article

Convergence Analysis of Contrastive Divergence Algorithm Based on Gradient Method with Errors

1Center for Intelligence Science and Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
2School of Mathematic and Information Science, Henan Polytechnic University, Jiaozuo, Henan 454000, China

Received 12 May 2015; Accepted 1 July 2015

Academic Editor: Julien Bruchon

Copyright © 2015 Xuesi Ma and Xiaojie Wang. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, 2006. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  2. A.-R. Mohamed, G. E. Dahl, and G. Hinton, “Acoustic modeling using deep belief networks,” IEEE Transactions on Audio, Speech and Language Processing, vol. 20, no. 1, pp. 14–22, 2012. View at Publisher · View at Google Scholar · View at Scopus
  3. A.-R. Mohamed, T. N. Sainath, G. E. Dahl, B. Ramabhadran, G. E. Hinton, and M. A. Picheny, “Deep belief networks using discriminative features for phone recognition,” in Proceedings of the 36th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '11), pp. 5060–5063, May 2011. View at Publisher · View at Google Scholar · View at Scopus
  4. V. Nair and G. E. Hinton, “3D object recognition with deep belief nets,” in Proceedings of the Neural Information Processing Systems Conference (NIPS '09), pp. 1339–1347, 2009.
  5. M. A. Salama, A. E. Hassanien, and A. A. Fahmy, “Deep Belief Network for clustering and classification of a continuous data,” in Proceedings of the 10th IEEE International Symposium on Signal Processing and Information Technology (ISSPIT '10), pp. 473–477, December 2010. View at Publisher · View at Google Scholar · View at Scopus
  6. F. Feng, R. Li, and X. Wang, “Deep correspondence restricted boltzmann machine for cross-modal retrieval,” Neurocomputing, vol. 154, pp. 50–60, 2015. View at Publisher · View at Google Scholar
  7. N. Jaitly and G. Hinton, “Learning a better representation of speech soundwaves using restricted boltzmann machines,” in Proceedings of the 36th IEEE International Conference on Acoustics, Speech, and Signal Processing, (ICASSP '11), pp. 5884–5887, May 2011. View at Publisher · View at Google Scholar · View at Scopus
  8. V. Mnih, H. Larochelle, and G. E. Hinton, “Conditional restricted Boltzmann machines for structured output prediction,” in Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI '11), F. G. Cozman and A. Pfeffer, Eds., p. 514, AUAI Press, 2011.
  9. A.-R. Mohamed and G. E. Hinton, “Phone recognition using restricted boltzmann machines,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '10), pp. 4354–4357, IEEE, Dallas, Tex, USA, March 2010. View at Publisher · View at Google Scholar · View at Scopus
  10. R. R. Salakhutdinov and G. E. Hinton, “Replicated soft-max: an undirected topic model,” in Advances in Neural Information Processing Systems (NIPS 2009), 2009. View at Google Scholar
  11. R. R. Salakhutdinov, A. Mnih, and G. E. Hinton, “Restricted Boltzmann machines for collaborative filtering,” in Proceedings of the 24th International Conference on Machine Learning (ICML '07), pp. 791–798, ACM, Corvallis, Ore, USA, June 2007. View at Publisher · View at Google Scholar · View at Scopus
  12. G. E. Hinton, “Training products of experts by minimizing contrastive divergence,” Neural Computation, vol. 14, no. 8, pp. 1771–1800, 2002. View at Publisher · View at Google Scholar · View at Scopus
  13. Y. Bengio and O. Delalleau, “Justifying and generalizing contrastive divergence,” Neural Computation, vol. 21, no. 6, pp. 1601–1621, 2009. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  14. S. Akoho and K. Takabatake, “Information geometry of contrastive divergence,” in Proceedings of the International Conference on Information Theory and Statistical Learning (ITSL '08), pp. 3–9, Las Vegas, Nev, USA, July 2008.
  15. I. Sutskever and T. Tieleman, “On the convergence properties of contrastive divergence,” in Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS '10), pp. 473–477, 789–795, May 2010.
  16. A. Yuille, “The convergence of contrastive divergences,” in Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS '04), pp. 1593–1600, December 2004. View at Scopus
  17. D. P. Bertsekas, Ed., Nonlinear Programming, Athena Scientic, Belmont, Mass, USA, 1995.
  18. D. P. Bertsekas and J. N. Tsitsiklis, “Gradient convergence in gradient methods with errors,” SIAM Journal on Optimization, vol. 10, no. 3, pp. 627–642, 2000. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  19. V. S. Borkar, “Asynchronous stochastic approximations,” SIAM Journal on Control and Optimization, vol. 36, no. 3, pp. 840–851, 1998. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  20. G. Pflug, “Optimization of stochastic models,” in The Interface between Simulation and Optimization, Kluwer, Boston, Mass, USA, 1996. View at Google Scholar
  21. H. Robbins and S. Monro, “A stochastic approximation method,” Annals of Mathematical Statistics, vol. 22, pp. 400–407, 1951. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at MathSciNet
  22. L. Younes, “On the convergence of Markovian stochastic algorithms with rapidly decreasing ergodicity rates,” Stochastics and Stochastics Reports, vol. 65, no. 3-4, pp. 177–228, 1999. View at Publisher · View at Google Scholar · View at MathSciNet
  23. G. R. Grimmett and D. R. Stirzaker, Probability and Random Process, Oxford University Press, 2001. View at MathSciNet
  24. A. Fischer and C. Igel, “Bounding the bias of contrastive divergence learning,” Neural Computation, vol. 23, no. 3, pp. 664–673, 2011. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus