Advances in Artificial Neural Systems

Research Article

A Simplified Natural Gradient Learning Algorithm

Table 2

LDPC experiment results.
LDPC OGL SNGL ANGL
Hidden units 10 10 10
Initial learning rate 0.2 0.2 0.01
Annealed learning rate No Yes No
Prior probability on parameters Yes Yes No
Parameter of prior distribution 0.001 0.001 N/A
Fisher information matrix size N/A
Adaptation rate N/A N/A
Number of extra parameters 0 200 28,900
Final learning rate 0.2 0.0097 2.6015
Minimum sum-squared error 0.3540 0.2691 0.00098
Final learning epoch 24,852 105 100,000