Table of Contents
Advances in Artificial Neural Systems
Volume 2011 (2011), Article ID 407497, 9 pages
http://dx.doi.org/10.1155/2011/407497
Research Article

A Simplified Natural Gradient Learning Algorithm

Department of Electrical and Computer Engineering, Utah State University, Logan, UT 84322, USA

Received 2 February 2011; Accepted 12 June 2011

Academic Editor: Shantanu Chakrabartty

Copyright © 2011 Michael R. Bastian et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Linked References

  1. S. I. Amari, “Natural gradient works efficiently in learning,” Neural Computation, vol. 10, no. 2, pp. 251–276, 1998. View at Google Scholar · View at Scopus
  2. S. I. Amari, H. Park, and K. Fukumizu, “Adaptive method of realizing natural gradient learning for multilayer perceptrons,” Neural Computation, vol. 12, no. 6, pp. 1399–1409, 2000. View at Google Scholar · View at Scopus
  3. H. Park, S. I. Amari, and K. Fukumizu, “Adaptive natural gradient learning algorithms for various stochastic models,” Neural Networks, vol. 13, no. 7, pp. 755–764, 2000. View at Publisher · View at Google Scholar · View at Scopus
  4. D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing, MIT Press, Cambridge, Mass, USA, 1986.
  5. D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press, New York, NY, USA, 2003.
  6. S. Bös and S. Amari, “Annealed online learning in multilayer neural networks,” in On-line Learning in Neural Networks, D. Saad, Ed., Cambridge University Press, New York, NY, USA, 1999. View at Google Scholar
  7. W. Wan, “Implementing online natural gradient learning: Problems and solutions,” IEEE Transactions on Neural Networks, vol. 17, no. 2, pp. 317–329, 2006. View at Publisher · View at Google Scholar · View at PubMed · View at Scopus
  8. S. Fiori, “Learning by natural gradient on noncompact matrix-type pseudo-riemannian manifolds,” IEEE Transactions on Neural Networks, vol. 21, no. 5, pp. 841–852, 2010. View at Publisher · View at Google Scholar · View at PubMed · View at Scopus
  9. M. K. Murray and J. W. Rice, Differential Geometry and Statistics, vol. 48 of Monographs on Statistics and Applied Probability, Chapman & Hall/CRC, London, UK, 1993.
  10. S. Amari and H. Nagaoka, Methods of Information Geometry, vol. 191 of Translations of Mathematical Monographs, Oxford University Press, New York, NY, USA, 2000.
  11. J. Nocedal and S. J. Wright, Numerical Optimization, Springer Series in Operations Research, Springer, New York, NY, USA, 1999.
  12. T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithmsfor Signal Processing, Prentice Hall, Upper Saddle River, NJ, USA, 1999.
  13. H. Park, N. Murata, and S. I. Amari, “Improving generalization performance of natural gradient learning using optimized regularization by NIC,” Neural Computation, vol. 16, no. 2, pp. 355–382, 2004. View at Publisher · View at Google Scholar · View at PubMed
  14. T. K. Moon, Error Correction Coding: Mathematical Methods and Algorithms, Wiley-Interscience, New York, NY, USA, 2005.