Research Article
Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN
Table 7
Patterns of hyperparameter effects and search direction.
| | Fold | Hidden nodes | Training cycles | Learning rate | Search direction |
| | 1 | Convex | Linear (−) | Linear (+) | Fix HC, TC at minimum MAE and increase TC | | 2 | Linear (−) | Convex | Linear (-) | Increase HN, LR together and fix TC at minimum MAE | | 3 | Convex | Linear (−) | Linear (−) | Fix HN at minimum MAE and increase TC and LR together | | 4 | Concave | Convex | Convex | Increase HN and fix TC | | 5 | Linear (−) | Convex | Linear (−) | Increase HN and LR together | | 6 | Convex | Convex | Linear (−) | Increase LR | | 7 | Linear (−) | Convex | Convex | Increase HN | | 8 | Linear (−) | Linear (−) | Convex | Increase HN and TC | | 9 | Linear (−) | Concave | Linear (+) | Increase HN | | 10 | Concave | Convex | Convex | Increase HN |
|
|
