Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

Table 5

Design and experiment results of Experiment 1.

Point type	Hyperparameter			MAE (validation)
	HN	TC	LR	Fold 1	Fold 2	Fold 3	Fold 4	Fold 5	Fold 6	Fold 7	Fold 8	Fold 9	Fold 10
1	1	28	0.00028	1.1332	1.1465	1.1254	1.0984	1.1071	1.0267	1.2692	1.1173	1.0874	1.1636
1	3	28	0.00028	1.1146	1.1502	1.1334	1.0932	1.0968	1.0202	1.2843	1.1294	1.0979	1.2028
1	1	82	0.00028	1.1271	1.1358	1.094	1.0736	1.0964	0.9994	1.2469	1.1184	1.064	1.1396
1	3	82	0.00028	1.0968	1.1301	1.079	1.076	1.0865	1.0024	1.2377	1.1203	1.0673	1.1515
1	1	28	0.00082	1.1943	1.1355	1.1634	1.0719	1.1013	1.0267	1.2563	1.1192	1.0741	1.1534
1	3	28	0.00082	1.1126	1.1306	1.0735	1.0753	1.0891	1.0202	1.2627	1.1166	1.0633	1.1422
1	1	82	0.00082	1.1785	1.1306	1.1399	1.0815	1.1001	0.9994	1.2433	1.1187	1.0757	1.1393
1	3	82	0.00082	1.1002	1.1338	1.0409	1.0936	1.0576	1.0024	1.2282	1.0857	1.0849	1.1382
−1	1	55	0.00055	1.1568	1.1344	1.1177	1.0728	1.0998	1.0091	1.2498	1.1186	1.0715	1.1446
−1	3	55	0.00055	1.0994	1.128	1.0669	1.0766	1.0821	1.0107	1.2496	1.1102	1.0669	1.1451
−1	2	10	0.00055	1.1334	1.1778	1.1729	1.1257	1.1336	1.0387	1.2884	1.164	1.1121	1.2212
−1	2	100	0.00055	1.0783	1.1275	1.0786	1.1046	1.0868	0.978	1.2232	1.1127	1.0793	1.1793
−1	2	55	0.0001	1.0783	1.1275	1.0786	1.1046	1.0868	0.978	1.2232	1.1127	1.0793	1.1793
−1	2	55	0.001	1.0852	1.1335	1.0873	1.1112	1.0974	0.9773	1.2438	1.1451	1.0833	1.2123
0	2	55	0.00055	1.0925	1.1483	1.1793	1.0804	1.0969	1.0005	1.2511	1.2337	1.1578	1.1438
0	2	55	0.00055	1.0925	1.1303	1.1046	1.1078	1.0865	0.9727	1.2771	1.2183	1.0833	1.1793
0	2	55	0.00055	1.0882	1.1433	1.1025	1.0948	1.0702	0.9904	1.2396	1.1337	1.0675	1.1806
0	2	55	0.00055	1.0947	1.1305	1.0741	1.0839	1.0886	1.0111	1.2367	1.1917	1.0742	1.1373
0	2	55	0.00055	1.2047	1.129	1.107	1.0889	1.1036	0.9756	1.2324	1.109	1.0673	1.143
0	2	55	0.00055	1.0777	1.1529	1.0663	1.09	1.096	0.9768	1.2828	1.1203	1.0822	1.1441