Ranking Biomedical Annotations with Annotator’s Semantic Relevancy
Table 1
Annotator and annotation predefined in the experiments.
UserType/Num
Details of the designed annotations
Annotation ratio
U1/200
All annotation are 100% correct, and 5 of them only contribute 5 or below annotations
15%
U2/300
40% annotation with correctness 0.95~1; 50% annotations with correctness 0.9~0.95; 10% annotations with correctness 0.85~0.9; 22 of them only contribute 5 or below annotations
30%
U3/200
15% annotation with correctness 0.95~1; 55% annotations with correctness 0.9~0.95; 20% annotations with correctness 0.85~0.9; 10% annotations with correctness 0.8~0.85; 30 of them only contribute 5 or below annotations
15%
U4/80
10% annotations with correctness 0.9~0.95; 60% annotations with correctness 0.85~0.9; 30% annotations with correctness 0.8~0.85
10%
U5/80
30% annotations with correctness 0.85~0.9; 40% annotations with correctness 0.8~0.85; 30% annotation with correctness 0.75~0.8
10%
U6/40
5% annotations with correctness 0.9~0.95; 20% annotations with correctness 0.85~0.9; 30% annotations with correctness 0.8~0.85; 30% annotation with correctness 0.75~0.8; 25% annotations with correctness 0.7~0.75
8%
U7/40
5% annotations with correctness 0.8~0.85; 15% annotations with correctness 0.75~0.8; 50% annotation with correctness 0.7~0.75; 30% annotations with correctness 0.6~0.7
7%
U8/30
10% annotations with correctness 0.75~0.8; 30% annotations with correctness 0.7~0.75; 60% annotation with correctness 0.6~0.7
3%
U9/30
All annotation are below 60% correct; 5 of them only contribute 5 or below annotations
