Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN

Table 11

Patterns of hyperparameter effects and search direction for dot and radial kernel function.

C	Conv	Search direction	dot	radial
Linear (−)	Linear (−)	Increase C and Conv together	Fold: 1, 5, 6, 10	Fold: 1
Concave	Linear (−)	Fix C at minimum MAE and increase Conv	Fold: 2, 7, 9
Linear (+)	Linear (−)	Fix C at minimum MAE and increase Conv	Fold: 3, 8
Linear (+)	Linear (+)	Fix C and Conv at minimum MAE	Fold: 4
Convex	Linear (+)	Fix C and Conv at minimum MAE		Fold: 2
Linear (+)	Linear (-)	Fix C at lower bound and increase Conv		Fold: 3
Concave	Linear (+)	Fix C and Conv at minimum MAE		Fold: 4, 5, 9
Linear (−)	Linear (+)	Increase C and fix Conv at minimum MAE		Fold: 7
Concave	Linear (−)	Fix C at lower bound and increase Conv		Fold: 8, 10