Research Article
Deep Neural Networks with Multistate Activation Functions
Table 2
Cross entropy losses on the training set. “Plain” means the conventional SGD without mean-normalisation and SVD restructuring. “MN” means mean-normalised SGD and “MN + SVD” means the SVD restructuring method using mean-normalised SGD.
| ||||||||||||||||||||||||||||||||