Hierarchical Self-Attention Hybrid Sparse Networks for Document Classification

Table 2

Sparse word encoder setting.

Sparse word encoder parameter	Setting
Dropout rate	0.1
RNN output size	50
Activate function	ReLU
Self-attention output size	100