Research Article
Online Learning for DNN Training: A Stochastic Block Adaptive Gradient Algorithm
| Input: | | Parameter: , and where and . denotes coordinate selection probability at time . Moreover, where and . | | Initially Set: and . | | Output: | (1) | fordo | (2) | | (3) | Generating diagonal matrix with probability | (4) | | (5) | Generating gradient | (6) | | (7) | | (8) | and | (9) | Clip | (10) | | (11) | | (12) | end for | (13) | return |
|