Deep Neural Networks with Multistate Activation Functions

Table 3

Cross entropy losses on the cross-validation set.

Activation function	Plain (%)	MN (%)	MN + SVD (%)
Logistic	1.7793	1.7648	1.7688
Two-order MSAF	1.7726	1.7743	1.7716
Three-order MSAF	1.7926	1.7733	1.7720
SYM-MSAF	1.7597	1.7539	1.7608