|
| Dataset | Experimenting | Dataset Name | Description |
|
| DS1 |
Experiment 1: content-based evaluation | Miller and Charles (MC)1 | RG consists of 65 pairs of nouns extracted from the WordNet, rated by multiple human annotators. |
| DS2 | Microsoft Research Paraphrase Corpus (MRPC)2 | The corpus consists of 5,801 sentence pairs collected from newswire articles, 3,900 of which were labeled as relatedness by human annotators. The whole set is divided into a training subset (4,076 sentences of which 2,753 are true) and a test subset (1,725 pairs of which 1,147 are true). |
|
| DS3 | Experiment 2: coselection-based evaluation (closest-synonym detection) | British National Corpus (BNC)3 | BNC is a carefully selected collection of 4124 contemporary written and spoken English texts, contains 100-million-word text corpus of samples of written and spoken English with the near-synonym collocations. |
| DS4 | SN (semantic neighbors)4 | SN relates 462 target terms (nouns) to 5910 relatum terms with 14.682 semantic relations (7341 are meaningful and 7341 are random). The SN contains synonyms coming from three sources: WordNet 3.0, Roget’s thesaurus, and a synonyms database. |
|
| DS5 | Experiment 2: coselection-based evaluation (semantic relationships exploration) | BLESS6 | BLESS relates 200 target terms (100 animate and 100 inanimate nouns) to 8625 relatum terms with 26.554 semantic relations (14.440 are meaningful (correct) and 12.154 are random). Every relation has one of the following types: hypernymy, cohypernymy, meronymy, attribute, event, or random. |
| DS6 | TREC5 | TREC includes 1437 sentences annotated with entities and relations at least one relation. There are three types of entities: person (1685), location (1968), and organization (978); in addition there is a fourth type others (705), which indicates that the candidate entity is none of the three types. There are five types of relations: located in (406) indicates that one location is located inside another location, work for (394) indicates that a person works for an organization, OrgBased in (451) indicates that an organization is based in a location, live in (521) indicates that a person lives at a location, and kill (268) indicates that a person killed another person. There are 17007 pairs of entities that are not related by any of the five relations and hence have the NR relation between them which thus significantly outnumbers other relations. |
| DS7 | IJCNLP 2011-New York Times (NYT)6 | NYT contains 150 business articles from NYT. There are 536 instances (208 positive, 328 negative) with 140 distinct descriptors in NYT dataset. |
| DS8 | IJCNLP 2011-Wikipedia8 | Wikipedia personal/social relation dataset was previously used in Culotta et al. [6]. There are 700 instances (122 positive, 578 negative) with 70 distinct descriptors in Wikipedia dataset. |
|
| DS9 | Experiment 3: task-based evaluation | Reuters 21,5787 | Reuters-21,578 contains 21,578 documents (12,902 are used) categorized to 10 categories. |
| DS10 | 20 Newsgroups8 | 20 newsgroups dataset contains 20,000 documents (18,846 are used) categorized to 20 categories. |
|
