Computational Intelligence and Neuroscience

Research Article

Gradient-Sensitive Optimization for Convolutional Neural Networks

Table 2

Parameter change in each algorithm during the iteration process in the Beale function test.
Iterations/methodsAdaGradAdamdiffGradRMSpropGS-AdamGS-RMSprop
0[0, 0][0, 0][0, 0][0, 0][0, 0][0, 0]
100[0.183, 0.2357][0.8557, 0.7214][0.893, 0.6537][1.0131, 0.4524][1.9231, 0.1244][2.1743, 0.2187]
200[0.2632, 0.3384][1.4836, 0.0146][1.5212, 0.0012][1.9186, 0.1407][2.2412, 0.2508][2.6504, 0.399]
300[0.3236, 0.42][1.906, 0.1342][1.9331, 0.144][2.6845, 0.4158][2.3906, 0.308][2.8522, 0.4608]
400[0.3737, 0.4935][2.1682, 0.2294][2.1876, 0.2366][2.9801, 0.5012][2.4847, 0.3426][2.9463, 0.4864]
500[0.417, 0.5629][2.3421, 0.2933][2.3568, 0.2987][2.9918, 0.5041][2.5519, 0.3663][2.9824, 0.4956]
600[0.4556, 0.6287][2.4663, 0.338][2.4781, 0.3421][2.9921, 0.5042][2.6035, 0.384][2.9935, 0.4984]
700[0.4902, 0.6904][2.5602, 0.3704][2.57, 0.3738][2.9922, 0.5042][2.6449, 0.3977][2.9986, 0.4996]
800[0.5217, 0.7464][2.6341, 0.395][2.6426, 0.3978][2.9922, 0.5042][2.6792, 0.4089][2.9999, 0.4999]
900[0.5504, 0.7955][2.6941, 0.4141][2.7015, 0.4165][2.9922, 0.5042][2.7082, 0.4181][2.9998, 0.5]
1000[0.5766, 0.8364][2.7435, 0.4294][2.7501, 0.4314][2.9922, 0.5042][2.7333, 0.4259][2.9999, 0.5]