Deep Neural Networks with Multistate Activation Functions

Table 1

Experimental results on TIMIT. “TR-XENT” means the cross entropy loss on the training set. “CV-XENT” means the cross entropy loss on the cross-validation set. “DEV-PER” means the PER on the development set. “TEST-PER” means the PER on the test set. “SYM-MSAF” means the symmetrical MSAF.

Activation function	TR-XENT	CV-XENT	DEV-PER (%)	TEST-PER (%)
Logistic	2.07	2.45	23.0	25.0
Two-order MSAF	1.89	2.29	21.8	24.0
Three-order MSAF	1.97	2.30	22.5	24.3
SYM-MSAF	1.91	2.24	22.1	23.6