Hierarchical Self-Attention Hybrid Sparse Networks for Document Classification

Table 3

Sparse sentence encoder setting.

Sparse sentence encoder parameter	Setting
Dropout rate	0.1
RNN output size	50
Activate function	ReLU
Self-attention output size	100
Kernel regularizer	L2