Table of Contents
Advances in Artificial Neural Systems
Volume 2011 (2011), Article ID 407497, 9 pages
http://dx.doi.org/10.1155/2011/407497
Research Article

A Simplified Natural Gradient Learning Algorithm

Department of Electrical and Computer Engineering, Utah State University, Logan, UT 84322, USA

Received 2 February 2011; Accepted 12 June 2011

Academic Editor: Shantanu Chakrabartty

Copyright © 2011 Michael R. Bastian et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Adaptive natural gradient learning avoids singularities in the parameter space of multilayer perceptrons. However, it requires a larger number of additional parameters than ordinary backpropagation in the form of the Fisher information matrix. This paper describes a new approach to natural gradient learning that uses a smaller Fisher information matrix. It also uses a prior distribution on the neural network parameters and an annealed learning rate. While this new approach is computationally simpler, its performance is comparable to that of adaptive natural gradient learning.