Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015, Article ID 267478, 12 pages
http://dx.doi.org/10.1155/2015/267478
Research Article

Topologically Ordered Feature Extraction Based on Sparse Group Restricted Boltzmann Machines

1School of Computer Science and Technology, Wuhan University of Technology, 122 Luoshi Road, Wuhan 430070, China
2State Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China
3Engineering Research Center for Spatio-Temporal Data Smart Acquisition and Application, Ministry of Education of China, Wuhan University, 129 Luoyu Road, Wuhan 430079, China
4Institute of Information Technology, Luoyang Normal University, 71 Luolong Road, Luoyang 471022, China

Received 20 March 2015; Revised 28 July 2015; Accepted 9 September 2015

Academic Editor: Panos Liatsis

Copyright © 2015 Zhong Chen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. Y. Freund and D. Haussler, “Unsupervised learning of distributions on binary vectors using two layer networks,” Tech. Rep., Computer Research Laboratory, University of California, Santa Cruz, Calif, USA, 1994. View at Google Scholar
  2. G. E. Hinton, “Training products of experts by minimizing contrastive divergence,” Neural Computation, vol. 14, no. 8, pp. 1771–1800, 2002. View at Publisher · View at Google Scholar · View at Scopus
  3. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, “A learning algorithm for boltzmann machines,” Cognitive Science, vol. 9, no. 1, pp. 147–169, 1985. View at Publisher · View at Google Scholar · View at Scopus
  4. G. F. Montúfar, J. Rauh, and N. Ay, “Expressive power and approximation errors of restricted Boltzmann machines,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '11), pp. 415–423, Granada, Spain, December 2011.
  5. G. E. Hinton and R. R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504–507, 2006. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  6. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, “Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations,” in Proceedings of the 26th International Conference On Machine Learning (ICML '09), pp. 609–616, June 2009. View at Scopus
  7. G. E. Hinton and R. Salakhutdinov, “Replicated softmax: an undirected topic model,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '09), pp. 1607–1614, 2009.
  8. A. Coates, A. Y. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS '11), pp. 215–223, Fort Lauderdale, Fla, USA, April 2011.
  9. M. Yasuda and K. Tanaka, “Approximate learning algorithm for restricted Boltzmann machines,” in Proceedings of the International Conference on Computational Intelligence for Modelling Control & Automation, pp. 692–697, IEEE, Vienna, Austria, December 2008. View at Publisher · View at Google Scholar
  10. P. Garrigues and B. Olshausen, “Learning horizontal connections in a sparse coding model of natural images,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '08), vol. 20, MIT Press, Cambridge, Mass, USA, 2008.
  11. H. Lee, C. Ekanadham, and A. Y. Ng, “Sparse deep belief net model for visual area v2,” in Proceedings of the 21st Annual Conference on Neural Information Processing Systems (NIPS '07), pp. 873–880, December 2007. View at Scopus
  12. H. Luo, R. Shen, C. Niu, and C. Ullrich, “Sparse group restricted Boltzmann machines,” in Proceedings of the 25th AAAI Conference on Artificial Intelligence, pp. 429–434, San Francisco, Calif, USA, August 2011.
  13. M. Welling, G. Hinton, and S. Osindero, “Learning sparse topographic representations with products of student-t distributions,” in Advances in Neural Information Processing Systems (NIPS), pp. 1359–1366, MIT Press, 2002. View at Google Scholar
  14. H. Goh, L. Kusmierz, J.-H. Lim, N. Thome, and M. Cord, “Learning invariant color features with sparse topographic restricted Boltzmann machines,” in Proceedings of the 18th IEEE International Conference on Image Processing (ICIP '11), pp. 1241–1244, IEEE, Brussels, Belgium, September 2011. View at Publisher · View at Google Scholar
  15. S. Bengio, F. Pereira, Y. Singer, and D. Strelow, “Group sparse coding,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '09), pp. 82–89, 2009.
  16. G. E. Hinton, “A practical guide to training restricted Boltzmann machines,” Momentum, vol. 9, no. 1, p. 926, 2010. View at Google Scholar
  17. J. Martens, A. Chattopadhya, T. Pitassi, and R. Zemel, “On the representational efficiency of restricted Boltzmann machines,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS '13), pp. 2877–2885, December 2013.
  18. A. Fischer and C. Igel, “Training restricted Boltzmann machines: an introduction,” Pattern Recognition, vol. 47, no. 1, pp. 25–39, 2014. View at Publisher · View at Google Scholar · View at Scopus
  19. H. Larochelle and Y. Bengio, “Classification using discriminative restricted Boltzmann machines,” in Proceedings of the 25th International Conference on Machine Learning (ICML '08), pp. 536–543, ACM, Helsinki, Finland, July 2008. View at Publisher · View at Google Scholar
  20. R. Salakhutdinov and H. Larochelle, “Efficient learning of deep Boltzmann machines,” in Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, vol. 6, Sardinia, Italy, May 2010.
  21. G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, no. 7, pp. 1527–1554, 2006. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus
  22. Y. Bengio and O. Delalleau, “Justifying and generalizing contrastive divergence,” Neural Computation, vol. 21, no. 6, pp. 1601–1621, 2009. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus
  23. Y. Bengio, “Learning deep architectures for AI,” Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1–27, 2009. View at Publisher · View at Google Scholar · View at Zentralblatt MATH · View at Scopus
  24. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, “Multimodal deep learning,” in Proceedings of the 28th International Conference on Machine Learning (ICML '11), pp. 689–696, Bellevue, Wash, USA, June-July 2011.
  25. Q. Zhou, L. Zhang, and L. Ma, “Learning topographic sparse coding through similarity function,” in Proceedings of the 4th International Conference on Natural Computation (ICNC '08), pp. 241–245, IEEE, Jinan, China, October 2008. View at Publisher · View at Google Scholar · View at Scopus
  26. F. Wu, Y. Yuan, Y. Rui, S. Yan, and Y. Zhuang, “Annotating web images using NOVA: non-convex group spArsity,” in Proceedings of the 20th ACM International Conference on Multimedia (MM '12), pp. 509–518, ACM, Nara, Japan, November 2012. View at Publisher · View at Google Scholar · View at Scopus
  27. L. Jacob, G. Obozinski, and J.-P. Vert, “Group lasso with overlap and graph lasso,” in Proceedings of the 26th International Conference On Machine Learning (ICML '09), pp. 433–440, June 2009. View at Scopus
  28. J. Mairal, R. Jenatton, G. Obozinski, and F. Bach, “Learning hierarchical and topographic dictionaries with structured sparsity,” in Wavelets and Sparsity XIV, Proceedings of SPIE, 81381P, p. 13, International Society for Optics and Photonics, 2011.
  29. T. Tieleman, “Training restricted boltzmann machines using approximations to the likelihood gradient,” in Proceedings of the 25th International Conference on Machine Learning, pp. 1064–1071, July 2008. View at Scopus
  30. K. Cho, A. Ilin, and T. Raiko, “Tikhonov-type regularization for restricted Boltzmann machines,” in Artificial Neural Networks and Machine Learning—ICANN 2012, vol. 7552 of Lecture Notes in Computer Science, pp. 81–88, Springer, Berlin, Germany, 2012. View at Publisher · View at Google Scholar
  31. P. O. Hoyer, “Non-negative matrix factorization with sparsity constraints,” Journal of Machine Learning Research, vol. 5, pp. 1457–1469, 2004. View at Google Scholar
  32. R. Zheng and J. Tai, Collected Edition of Flying Apsaras in the Dunhuang Grotto Murals, China, Commercial Press, Hong Kong, 2002 (Chinese).
  33. Z. Chen, S. Xiong, Z. Fang, Q. Li, B. Wang, and Q. Zou, “A kernel support vector machine-based feature selection approach for recognizing Flying Apsaras' streamers in the Dunhuang Grotto Murals, China,” Pattern Recognition Letters, vol. 49, pp. 107–113, 2014. View at Publisher · View at Google Scholar · View at Scopus
  34. N. Hurley and S. Rickard, “Comparing measures of sparsity,” IEEE Transactions on Information Theory, vol. 55, no. 10, pp. 4723–4741, 2009. View at Publisher · View at Google Scholar · View at MathSciNet · View at Scopus