Research Article
On the Brittleness of Handwritten Digit Recognition Models
Table 4
Dataset independence for gradient-based features, two datasets combined.
| Classifier | Trained on | Tested on | Error versus avg. of own testsets | MNIST | DIGITS | USPS |
| IBk1 eucl. | MNIST-DIGITS | 3.58 | 7.41 | 9.87 | 1.80x | IBk1 eucl. | MNIST-USPS | 1.98 | 15.59 | 3.34 | 5.86x | IBk1 eucl. | USPS-DIGITS | 9.51 | 7.80 | 4.14 | 1.59x |
| SVM linear | MNIST-DIGITS | 3.43 | 4.79 | 11.01 | 2.68x | SVM linear | MNIST-USPS | 2.23 | 15.14 | 3.39 | 5.39x | SVM linear | USPS-DIGITS | 7.50 | 6.24 | 4.38 | 1.41x |
| SVM poly. | MNIST-DIGITS | 1.84 | 3.51 | 6.58 | 2.46x | SVM poly. | MNIST-USPS | 0.91 | 10.75 | 2.54 | 6.23x | SVM poly. | USPS-DIGITS | 5.94 | 4.45 | 2.94 | 1.61x |
| SVM RBF | MNIST-DIGITS | 1.88 | 3.51 | 6.98 | 2.59x | SVM RBF | MNIST-USPS | 1.00 | 11.53 | 2.44 | 6.70x | SVM RBF | USPS-DIGITS | 6.22 | 4.57 | 2.84 | 1.68x |
|
|