Key -Gram Extractions and Analyses of Different Registers Based on Attention Network

Figure 1

The overview of the attentive -gram network. As an example, the input sentence is s_i = {这是我妹妹。}. The embedding layer converts the words in into the corresponding vector . Then, -gram vectors come from the concatenation of these word vectors. In the figure, as an example, the 2-gram vectors come from the concatenation of every two adjacent word vectors. When , the attention layer scores each word in the sentence to obtain the score . Similarly, when , the score corresponding to each 2-gram in is . Weighting the sum of all by the weights , we obtain the sentence vector. In the concatenation layer, and are concatenated together to get the sentence vector , which is the input to the MLP classifier. After classification layers, we get the output, probabilities of the sentence belonging to each category. The symbols are described in detail in Section 2.1. The attentive -gram network (ANN) structure with 1,2-grams.