Pipelined Training with Stale Weights in Deep Convolutional Neural Networks

Figure 8

Memory increase vs batch size for 4-stage pipelined training. (a) LeNet-5 on MNIST. (b) AlexNet on CIFAR-10. (c) VGG-16 on CIFAR-10. (d) ResNet-20 on CIFAR-10.

(a)

(b)

(c)

(d)