Research Article

Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

Table 14

Patterns of hyperparameter effects and search direction for DBN.

FoldHNLRRBMnniLRNbSearch direction

1Linear (−)ConcaveLinear (−)Linear (+)Linear (−)Linear (+)Increase HN, N, LR, and fix LRRBM, ni, Nbat min. MAE
2ConvexLinear (+)Linear (−)Linear (+)Linear (−)Linear (+)Increase N, LR, and fix HN, LRRBM, ni, Nbat min. MAE
3Linear (−)Linear (−)Linear (−)Linear (+)Linear (−)Linear (+)Increase HN, LRRBM, N, LR, and fix ni, Nbat min. MAE
4Linear (−)Linear (−)Linear (−)Linear (−)Linear (−)Linear (+)Increase HN, LRRBM, N, ni, LR, and fix, Nbat min. MAE
5Linear (+)Linear (−)Linear (−)Linear (−)Linear (−)Linear (+)Increase LRRBM, N, ni, LR, and fix HN, Nbat min. MAE
6Linear (−)Linear (+)Linear (−)Linear (−)Linear (−)Linear (+)Increase HN, N, ni, LR, and fix LRRBM, Nbat min. MAE
8Linear (−)ConvexLinear (−)ConvexLinear (−)ConcaveIncrease HN, N, LR, and fix LRRBM, ni, Nbat min. MAE
9Linear (−)Linear (−)Linear (−)Linear (+)Linear (−)Linear (+)Increase HN, LRRBM, N, LR, and fix ni, Nbat min. MAE
10Linear (+)Linear (+)Linear (−)Linear (+)Linear (−)Linear (+)Increase N, LR, and fix HN, LRRBM, ni, Nbat min. MAE