Chinese Clinical Named Entity Recognition with ALBERT and MHA Mechanism

Table 4

Summary of different optimizers.

Optimizer	Year	Learning rate	Gradient
AdaGrad	2011	√	×
RMSprop	2012	√	×
Adam	2014	√	√
Lookahead + Adam	2019	√	√
RAdam	2019	√	√

“√” means dynamic adjustment, “×” means not.