Research Article
Pipelined Training with Stale Weights in Deep Convolutional Neural Networks
Table 7
Memory usage of 4-stage pipelined ResNet training.
| | CNN | PPV | Activations | Weights (MB) | Increase | Increase percentage in upper bound |
| | ResNet-56 | (19) | 10.87 MB × batch size | 3.25 | 6.32 MB × batch size | 58 | | ResNet-110 | (37) | 21.43 MB × batch size | 6.59 | 12.35 MB × batch size | 57 | | ResNet-224 | (75) | 43.70 MB × batch size | 13.64 | 25.07 MB × batch size | 57 | | ResNet-362 | (121) | 70.67 MB × batch size | 22.17 | 40.50 MB × batch size | 57 |
|
|
