Research Article
Pipelined Training with Stale Weights in Deep Convolutional Neural Networks
Algorithm 1
Pipelined stochastic gradient descent (PPL-SGD).
| Initialize weights | | Given learning rate sequence | | fortodo | | fortoin parallel do | | Compute gradient using stale weights: | | | | Update weights: | | | | end for | | end for |
|