A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs

Table 4

Average predictive accuracy and standard deviations of the extracted rules for boosted shallow trees trained by modest boosting and gentle boosting.

Dataset	BST-M1	BST-M2	BST-M3	BST-M4	BST-G1	BST-G2	BST-G3	BST-G4
Australian Credit Appr.	85.6 0.2	85.8 0.3	86.0 0.7	86.1 0.9	84.8 0.8	85.2 1.1	85.3 0.8	85.1 0.6
Breast Cancer	95.9 0.2	96.3 0.4	96.5 0.4	96.2 0.2	95.8 0.3	95.9 0.5	96.1 0.5	96.3 0.4
Breast Cancer 2	95.6 0.6	95.7 0.5	95.1 0.8	95.1 0.7	96.4 0.7	96.0 0.7	95.8 0.6	95.7 0.4
Breast Canc. (prognostic)	73.8 1.6	72.4 1.7	72.9 2.1	72.9 2.6	72.7 2.4	75.0 1.6	74.7 2.8	74.6 1.8
Bupa Liver Disorders	64.5 1.4	69.5 2.2	70.9 1.5	71.1 1.6	70.5 2.3	68.8 1.6	67.0 2.2	67.2 1.2
Chess (kr-versus-kp)	93.5 0.0	95.1 0.2	96.2 0.1	97.5 0.2	96.7 0.1	99.5 0.1	99.8 0.1	99.7 0.1
Coronary Heart Disease	80.5 0.7	88.8 0.4	89.2 0.7	89.4 0.3	87.7 0.7	89.2 0.9	89.0 0.6	89.1 0.9
German Credit	72.3 0.4	72.5 0.4	72.5 0.4	72.4 0.7	74.2 0.7	73.8 1.2	73.3 1.1	73.1 0.6
Glass (binary)	79.9 1.4	84.6 1.5	85.4 1.9	85.2 1.7	82.9 1.8	83.1 2.3	83.3 2.3	84.1 1.5
Haberman	75.0 1.0	74.4 0.7	74.5 0.9	74.6 1.0	72.5 1.8	66.5 1.5	64.9 2.4	65.3 1.5
Heart Disease	83.1 0.8	81.6 1.9	80.7 2.0	80.8 1.8	80.6 1.6	77.1 2.4	76.4 1.8	76.5 2.0
ILPD (liver)	70.5 0.7	69.9 0.8	69.1 0.6	69.4 0.7	69.5 1.2	69.7 1.4	69.6 1.5	69.6 1.6
Ionosphere	92.1 1.1	91.2 0.8	91.8 0.6	91.5 1.1	91.5 0.9	90.5 1.3	91.3 0.8	91.4 1.2
Istanbul Stock Exch.	77.1 0.7	75.8 0.7	76.6 0.9	76.4 0.8	75.1 1.1	73.2 1.2	72.5 1.5	72.8 1.2
Labor	83.6 1.7	84.2 2.8	87.5 4.1	84.7 3.3	81.9 2.9	86.0 3.8	84.4 3.3	82.9 4.0
Musk1	73.1 0.7	83.5 1.5	84.9 1.4	85.3 1.4	83.4 1.5	86.0 1.6	86.7 1.5	87.2 1.8
Pima Indians	75.0 0.5	76.0 0.9	75.7 0.5	75.9 0.7	75.0 0.9	72.8 0.9	72.0 1.1	72.3 1.0
Promoters	84.7 3.1	79.7 3.4	81.2 3.5	78.1 5.2	82.8 4.3	80.4 4.8	80.6 3.3	81.5 1.6
Saheart	71.4 1.2	72.2 1.4	72.3 1.1	71.7 1.3	67.2 1.1	64.5 1.8	65.3 1.8	63.0 1.5
Sonar	80.1 2.0	79.4 2.5	81.9 1.6	80.8 2.5	80.9 1.7	81.8 2.7	80.1 3.8	81.4 1.5
Spect Heart	71.0 1.2	72.1 1.0	71.4 1.3	71.6 1.7	71.4 2.0	65.4 1.8	66.1 2.0	66.3 1.9
Splice junct.	89.4 0.4	96.0 0.1	97.0 0.2	97.1 0.2	94.8 0.3	96.7 0.2	96.4 0.2	95.9 0.3
Svmguide	96.8 0.1	97.1 0.1	97.1 0.1	97.2 0.1	97.0 0.1	97.0 0.1	96.8 0.2	96.8 0.1
Tictactoe	70.0 0.2	73.5 0.4	83.2 0.7	95.1 0.9	98.3 0.1	99.3 0.2	99.8 0.2	100.0 0.0
Vertebral Column	79.2 0.7	82.8 1.6	82.3 1.5	81.8 1.0	81.6 1.6	80.3 1.2	81.0 1.1	80.6 1.5