Fed-DNN-Debugger: Automatically Debugging Deep Neural Network Models in Federated Learning

Table 4

Debugging a single model.

Dataset	Models	Problem type	Origin		MODE			Fed-DNN-Debugger			Improvement
			MAcc (%)	LAcc (%)	Time (s)	MAcc (%)	LAcc (%)	Time (s)	MAcc (%)	LAcc (%)	Faster	MAcc (%)	LAcc (%)
MNIST	Model-1	Overfitting	91.11	81.3	169.11	96.65	94.6	11.81	97.96	98.7	14.32	1.31	4.1%
		Underfitting	93.00	82.5	118.86	96.82	97.6	8.18	97.91	99.2	14.53	1.09	1.6%
	Model-2	Overfitting	92.42	81.4	144.96	92.42	81.4	15.2	98.16	97.7	9.53	5.74	16.3%
		Underfitting	93.88	87.0	194.45	96.96	96.6	10.84	97.81	97.0	17.94	0.85	0.4%
	Model-3	Overfitting	88.65	73.5	490.62	96.63	93.2	31.25	97.61	97.3	15.70	0.98	4.1%
		Underfitting	91.54	83.9	566.26	97.63	98.9	30.65	98.62	99.0	18.47	0.99	0.1%
	Model-4	Overfitting	94.89	88.3	863.60	96.97	95.3	191.03	98.14	96.2	4.52	1.17	0.87%
		Underfitting	88.17	83.9	1523.38	95.99	98.2	543.27	96.73	98.9	2.80	0.74	0.75%
CIFAR-10	Model-1	Overfitting	66.51	49.7	251.67	67.73	51.1	7.1	69.72	61.9	35.44	1.99	10.8%
		Underfitting	68.07	48.8	187.31	69.30	41.5	6.9	72.40	68.6	27.14	3.10	27.1%
	Model-2	Overfitting	76.94	57.0	459.84	77.11	76.6	8.12	79.21	76.8	56.63	2.10	0.2%
		Underfitting	72.04	56.5	330.79	74.28	62.0	7.96	76.88	66.3	41.55	2.60	4.33%
	Model-3	Overfitting	80.57	65.4	743.02	82.66	78.3	13.92	88.23	86.7	53.38	5.57	8.4%
		Underfitting	75.66	60.1	584.86	82.14	75.3	14.22	87.89	86.2	41.13	5.75	10.87%
	Model-4	Overfitting	77.52	65.5	494.04	85.04	85.9	148.77	88.16	87.9	3.32	3.12	2.08%
		Underfitting	84.58	63.7	1359.60	87.26	87.2	88.96	89.00	87.3	15.28	1.74	0.11%
Average improvement											23.23	2.42	5.76%