Gradient-Sensitive Optimization for Convolutional Neural Networks

Table 4

Error rates of sample classification on the testing data.

Group/algorithm	1	2	3	4	5
AdaGrad	0.0384	0.0396	0.0351	0.0476	0.0424
Adam	0.0170	0.0173	0.0158	00188	0.0187
RMSprop	0.0223	0.0217	0.0276	0.0198	0.0191
diffGrad	0.0213	0.0256	0.0235	0.0224	0.0269
AdaHMG	0.0186	0.0181	0.0214	0.0219	0.0191
GS-Adam	0.0150	0.0170	0.0169	0.0161	0.0168
GS-RMSprop	0.0151	0.0157	0.0156	0.0164	0.0165

The lowest error rate in each test is indicated in bold.