Improving Predictions of Multiple Binary Models in ILP

Table 3

Accuracies of our new multi-class methods (MRL, MRSU, and MRSI) compared to CN2 accuracy, with average ranks in brackets. The 6th column shows the multi-model accuracy as reported by Aleph, which is particularly optimistic for multi-class problems due to overemphasising the default rules. The rightmost column shows the average positive recall, which ignores the default rules but is still not equal to multi-class accuracy as conflicting predictions are not taken into account.

	MRL	MRSU	MRSI	CN2	Aleph standard
	Multi-class accuracy				Multi-model accuracy	Average recall
Data set 1	81.43 (2.00)	81.32 (4.00)	83.98  (1.00)	81.38  (3.00)	86.90	82.18
Data set 2	83.70 (2.00)	83.55 (3.50)	84.86  (1.00)	83.55  (3.50)	91.52	82.91
Data set 3	90.43  (1.00)	86.77  (4.00)	89.75  (2.00)	88.92  (3.00)	90.27	86.46
Data set 4	60.67 (3.00)	58.00 (4.00)	64.00  (1.00)	62.67  (2.00)	72.44	48.00
Data set 5	80.48 (3.00)	80.69 (2.00)	79.70 (4.00)	80.94  (1.00)	89.91	70.82
Data set 6	80.64 (2.00)	72.51 (4.00)	83.68  (1.00)	76.20  (3.00)	79.04	71.20
Average	79.56 (2.17)	77.14 (3.58)	80.99  (1.67)	78.94  (2.58)	85.01	73.59
Data set 7	77.06  (2.00)	77.06 (2.00)	76.55  (4.00)	77.06  (2.00)	73.97	73.97
Data set 8	60.18 (4.00)	60.91 (3.00)	65.38  (1.00)	60.98  (2.00)	77.06	77.06
Data set 9	78.24  (1.00)	76.07 (3.00)	77.14  (2.00)	75.55  (4.00)	60.11	60.18
Data set 10	76.56  (2.00)	76.56 (2.00)	76.48  (4.00)	76.56  (2.00)	76.56	76.56
Data set 11	74.95 (3.00)	75.02 (2.00)	74.59 (4.00)	75.09  (1.00)	74.80	74.80
Average	73.40 (2.40)	73.12 (2.40)	74.03 (3.00)	73.05 (2.20)	72.50	72.51