On the Brittleness of Handwritten Digit Recognition Models

Table 2

Dataset independence for gradient-based features, each dataset separately.

Classifier	Trained on	Tested on			Avg. error versus own testset
		MNIST	DIGITS	USPS
IBk1 euclidean	MNIST	1.29	12.08	5.98	7.00x
IBk1 euclidean	DIGITS	21.91	7.29	37.62	4.08x
IBk1 euclidean	USPS	10.30	33.07	3.49	6.21x
SVM linear	MNIST	1.34	12.92	5.63	6.92x
SVM linear	DIGITS	19.76	5.12	30.54	4.91x
SVM linear	USPS	14.62	39.37	3.34	8.08x
SVM polynomial	MNIST	0.47	8.07	4.43	13.30x
SVM polynomial	DIGITS	17.81	3.67	24.86	5.81x
SVM polynomial	USPS	14.68	39.03	2.79	9.63x
SVM RBF	MNIST	0.57	8.30	4.28	11.04x
SVM RBF	DIGITS	17.75	4.06	25.46	5.32x
SVM RBF	USPS	14.89	40.03	2.79	9.84x