Fed-DNN-Debugger: Automatically Debugging Deep Neural Network Models in Federated Learning

Table 2

Debugging federated models.

Dataset	Models	Problem type	Origin		FedAvg [1]		Fed-DNN-Debugger		Improvement
			MAcc (%)	LAcc (%)	MAcc (%)	LAcc (%)	MAcc (%)	LAcc (%)	MAcc (%)	LAcc (%)
MNIST	Model-1	Overfitting	91	81.31	94.53	94.12	96.1	97.95	1.57	3.83
		Underfitting	93	82.48	94.38	92.87	98.3	98.98	3.92	6.11
	Model-2	Overfitting	92.42	81.42	95.88	95.38	96.39	96.71	0.51	1.33
		Underfitting	93.88	86.97	95.44	94.17	98.37	98.27	2.93	4.1
	Model-3	Overfitting	88.65	73.5	94.59	92.99	96.4	97.23	1.81	4.24
		Underfitting	91.54	86.35	95.85	94.05	97.77	98.88	1.92	4.83
	Model-4	Overfitting	94.89	88.30	96.69	94.36	97.23	98.15	0.54	3.79
		Underfitting	88.17	85.87	96.72	95.44	97.86	98.54	1.14	3.1
CIFAR-10	Model-1	Overfitting	66.51	49.7	64.81	45.4	68.61	64.48	3.8	19.08
		Underfitting	68.07	48.8	69.58	56.4	72.75	70.4	3.17	14
	Model-2	Overfitting	70.94	61	71.87	62.2	72.91	71.49	1.04	9.29
		Underfitting	72.04	50.8	68.07	67.9	76.51	76.1	8.44	8.2
	Model-3	Overfitting	80.57	68.67	82.85	82.89	84.86	84.68	2.01	1.79
		Underfitting	75.66	65.9	75.66	75.2	82.56	79.9	6.9	4.7
	Model-4	Overfitting	77.52	66.3	78.68	73.2	80.05	76.61	1.37	3.41
		Underfitting	84.58	74.1	85.22	78	86.95	84.7	1.73	6.7
Average improvement									2.68	5.21