Providing Definitive Learning Direction for Relation Classification System

Table 2

Comparison with previous relation classification systems on SemEval-2010 Task 8 benchmark. Symbol “∘” means the experimental result is implemented by us. In the last line of the form, the performance of two strategies described in Section 3.4 is presented; they are represented as Max-pooling and Sum.

Model	Additional information	-score
SVM; (Rink and Harabagiu 2010)	POS, Prefixes, Morphological, WordNet, Dependency Parse,; Levin Classed, ProBank, FrameNet, NomLex-Plus,; Google N-Gram, Paraphrases, TextRunner	82.2
MVRNN; (Socher et al. 2012)	Word embedding, syntactic parsing tree; +POS, NER, WordNet	79.1; 82.4
CNN; (Zeng et al. 2014)	Word embedding, position feature; +WordNet, words around nominal	78.9; 82.7
BRNN; (Zhang and Wang 2015)	Word embedding	82.5
CR-CNN; (Santos et al. 2015)	Word embedding; +position feature	82.8; 84.1
SDP-LSTM; (Xu et al. 2015)	Word embedding; +POS, GR, WordNet embedding	82.4; 83.7
BLSTM; (Zhang et al. 2015)	Word embedding; +position feature, POS, NER, WNSYN, DEP	82.7; 84.3
Att-BLSTM; (Zhou et al. 2016); Att-BLSTM^∘	Word embedding, Position Indicator; Word embedding, position feature	84.0; 83.5
Bi-GRU + InConcat	Word embedding, position feature	83.9
Bi-GRU + OutConcat	Word embedding, position feature	84.6
Bi-GRU + EAtt + Max-pooling	Word embedding, position feature	83.5
Bi-GRU + EAtt + Sum	Word embedding, position feature	84.7