Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

Table 15

Best hyperparameter setting and the validation performance of DBN using RSM.

Cross-validation								Confirmation runs
Fold	HN	ni	n	LRRBM	LR	Nb	MAE	Avg MAE	SD (MAE)	95% PI half-width	PI coverage
1	7	389	10	0.0001	0.0069	1025	1.081	1.097	0.032	0.109	Yes
2	3	530	10	0.001	0.0107	1025	1.166	1.168	0.009	0.03	Yes
3	8	575	10	0.001	0.0067	350	1.073	1.083	0.011	0.036	Yes
4	3	254	10	0.001	0.0068	1025	1.083	1.096	0.013	0.042	Yes
5	3	209	10	0.001	0.0073	1700	1.087	1.093	0.009	0.031	Yes
6	5	209	10	0.001	0.008	1025	1.000	0.998	0.013	0.045	Yes
7	4	434	10	0.001	0.0094	1025	1.223	1.232	0.018	0.061	Yes
8	7	344	10	0.001	0.0071	1025	1.12	1.14	0.035	0.118	Yes
9	6	620	10	0.001	0.0101	1025	1.131	1.089	0.028	0.095	Yes
10	6	620	10	0.001	0.0101	350	1.299	1.203	0.064	0.215	Yes

Bold values emphasize that this is the combination of hyperparameters that results in the minimal average MAE (0.998).