On the Brittleness of Handwritten Digit Recognition Models

Table 4

Dataset independence for gradient-based features, two datasets combined.

Classifier	Trained on	Tested on			Error versus avg. of own testsets
		MNIST	DIGITS	USPS
IBk1 eucl.	MNIST-DIGITS	3.58	7.41	9.87	1.80x
IBk1 eucl.	MNIST-USPS	1.98	15.59	3.34	5.86x
IBk1 eucl.	USPS-DIGITS	9.51	7.80	4.14	1.59x
SVM linear	MNIST-DIGITS	3.43	4.79	11.01	2.68x
SVM linear	MNIST-USPS	2.23	15.14	3.39	5.39x
SVM linear	USPS-DIGITS	7.50	6.24	4.38	1.41x
SVM poly.	MNIST-DIGITS	1.84	3.51	6.58	2.46x
SVM poly.	MNIST-USPS	0.91	10.75	2.54	6.23x
SVM poly.	USPS-DIGITS	5.94	4.45	2.94	1.61x
SVM RBF	MNIST-DIGITS	1.88	3.51	6.98	2.59x
SVM RBF	MNIST-USPS	1.00	11.53	2.44	6.70x
SVM RBF	USPS-DIGITS	6.22	4.57	2.84	1.68x